We would like to thank the Vodafone Americas Foundation, the Bill and Melinda Gates Foundation, and the S&#227;o Paulo Research Foundation (FAPESP) for funding this research. We would also like to thank the many faculty members from the Department of Entomology at University of California, Riverside, for their advice on this project.&#160;

1. Insect Colony and Rearing
<ol>
	<li>Mosquito Colony and Rearing
	<ol>
		<li>Rear Culex tarsalis, Culex quinquefasciatus, Culex stigmatosoma, and Aedes aegypti adults from lab colonies, which originated from wild caught individuals.</li>
		<li>Rear mosquito larvae in enamel pans under standard laboratory conditions (27 &#176;C, 16:8 hr light:dark [LD] cycle with 1 hr dusk/dawn periods), and feed them ad libitum on a mixture of ground rodent chow and Brewer&#8217;s yeast (3:1, v:v).</li>
		<li>Collect mosquito pupae into 200&#160;ml&#160;cups, and place them into experimental chambers. Alternatively, aspirate the adult mosquitoes into experimental chambers within 1 week of emergence. Make sure each experimental chamber contains 20 to 40 individuals of the same species/sex.</li>
		<li>Feed adult mosquitoes ad libitum on a 10% sucrose and water mixture. Replace food weekly.</li>
		<li>Moisten cotton towels twice a week and place them on top of the experimental chambers to maintain humidity within the cage. In addition, place a 200&#160;ml cup of tap water in the chamber at all times to help maintain the overall humidity level.</li>
		<li>Maintain the experimental chambers on a 16:8 hr light:dark [LD] cycle, 20.5-22 &#176;C and 30-50% RH for the duration of the experiment.</li>
	</ol>
	</li>
	<li>House Fly and Fruit Fly Colony and Rearing
	<ol>
		<li>Rear Musca&#160;domestica from a lab colony, derived from wild caught individuals. Catch wild Drosophila&#160;simulans individuals and rear them in the experimental chambers.</li>
		<li>Rear Musca domestica larvae in plastic tubs under standard laboratory conditions (12:12 hr light:dark [LD] cycle, 26 &#176;C, 40% RH) in a mixture of water, bran meal, alfalfa, yeast, and powdered milk. Rear Drosophila&#160;simulans larvae in a rearing chamber and feed them ad libitum on a mixture of rotting fruit.</li>
		<li>Aspirate adult Musca&#160;domestica into experimental chambers within 1 week of emergence. Adult Drosophila&#160;simulans can be directly reared in the experimental chambers. Prior to data collection, make sure each experimental chamber contains no more than 10-15 individual Musca domestica or 20-30 individual Drosophila simulans.</li>
		<li>Feed adult Musca&#160;domestica ad libitum on a mixture of sugar and low-fat dried milk, with free access to water. Feed adult Drosophila&#160;simulans&#160;ad libitum on a mixture of rotting fruit. Replace food weekly.</li>
		<li>Maintain experimental chambers on a 16:8 hr light:dark [LD] cycle, 20.5-22 &#176;C and 30-50% RH for the duration of the experiment.</li>
	</ol>
	</li>
</ol>
2. Record Flying Sounds in Experimental Chambers
<ol>
	<li>Experimental Chamber Setup 
	Note: An &#8220;experimental chamber&#8221; denotes the cage designed in our lab, in which the data was recorded. The sensor is fairly inexpensive. When built in bulk, a set up could be manufactured for less than $10.
	<ol>
		<li>Construct an experimental chamber, either of the larger size: 67 cm L x 22 cm W x 24.75 cm H, or the smaller size: 30 cm L x 20 cm W x 20 cm H. The experimental chamber consists of a phototransistor array and a laser line pointing at the phototransistor array. 
		NOTE: Additionally, the chamber consists of Kritter Keepers that are modified to include the sensor apparatus as well as a sleeve attached to a piece of PVC piping to allow access to the insects.</li>
		<li>Connect the phototransistor array to an electronic board. The output of the electronic board feeds into a digital sound recorder and is recorded as audio data in the MP3 format. See the logic design of the sensor in Figure 1.II and a physical version of the chamber in Figure 1.I.</li>
		<li>Modify the lids of the experimental chambers with a piece of mesh cloth affixed to the inside in order to prevent escape of the insects. 
		NOTE: When an insect flies across the laser beam, its wings partially occlude the light, causing small light fluctuations. The light fluctuations are captured by the phototransistor array as changes in current, and the signal is filtered and amplified by the custom designed electronic board.</li>
	</ol>
	</li>
	<li>Set Up the System to Record Sounds Produced by Flying Insects
	<ol>
		<li>Connect the experimental chamber to a power supply. Turn on the power.</li>
		<li>On the experimental chamber, find the laser lights and photoarray. Align the laser lights to the photoarray. To achieve the proper alignment, adjust the photoarray using the magnets on the outside of the experimental chamber which correspond to the magnets attached to the photoarray on the interior of the chamber until the laser light is centered on all the individual photodiodes.</li>
		<li>Perform two sanity checks to make sure the system is properly set up. 
		Note: The first step is to make sure that the system is powered, all wires are properly connected and the laser is pointing at the photo array. The second step is to conduct further checks on the laser alignment to ensure it can capture the sound of the insects&#8217; wingbeats.
		<ol>
			<li>Plug headphones (rather than the recorder) into the audio jack. Plunge hand in and out of the experimental chamber, near the photoarray, to break the plane of the laser light. Make sure the laser light is on (it will be a red beam of light) and that you break the plane of light a few times with your hand. Listen for changes in noise level as your hand goes in and out of the light beam. If you detect an audible difference, the sensor is able to capture sounds produced by the movement of big objects. If successful, move on to the next sanity check, otherwise, check if the headphone is properly connected and whether the laser is pointing at the photoarray. Adjust the photoarray accordingly until the sound of the hand moving in and out of the experimental chamber can be heard.</li>
			<li>Attach a thin piece of electrical wiring to an automatic toothbrush. Turn on the toothbrush, and plunge the wiring in and out of the experimental chamber close to the phototarray. Make sure the laser light is hitting the piece of wiring as it moves. If you detect an audible change in frequency, when the piece of wiring breaks the plane of laser light, the system is then ready to capture the sounds produced by the movement of tiny objects, i.e., insect sounds. If you do not detect an audible difference, go back to step 2.2.2 to re-align the laser lights and the photoarray.</li>
		</ol>
		</li>
		<li>After the system is properly set up, close the lid and add the insects.</li>
	</ol>
	</li>
	<li>Data Collection: Record Sounds Produced by Flying Insects
	<ol>
		<li>Turn on the recorder and make a voice annotation that includes the following information: name of the species in the experimental chamber, age of the insects, date and time, current ambient RT, and relative humidity. Pause the recording.</li>
		<li>Connect the recorder to the system, via the audio cable, and resume the recording. Leave the recorder to record for 3 days, then stop the recording.</li>
		<li>Download the data from the recorder into a new folder on a PC. Empty the recorder by deleting the data.</li>
		<li>Repeat the above recording process, until the remaining insects have died off and there are no more than 5 insects left alive in the cage.</li>
	</ol>
	</li>
</ol>
3. Sensor Data Processing and Detection of Sounds Produced by Flying Insects
<ol>
	<li>Use Software to Detect Sounds Produced by Flying Insects. 
	Note: The software (detection algorithm) is much faster than real-time. It takes less than 3&#160;hr&#160;to process a recording session, i.e., three days of data, on a standard machine with Intel(R) Core&#8482; CPU at 2.00 GHz and 8 GB RAM.
	<ol>
		<li>For each folder containing data from a recording session, run the detection software to detect insect sounds. To run the software, open MATLAB, and type &#8220;circandian_wbf (dataDir)&#8221; in the command window, where dataDir is the directory of the recording data. Then press &#8220;Enter&#8221; to start. 
		NOTE: Download the detection software circadian_wbf from reference #16.</li>
		<li>Wait until the algorithm terminates, then check the detection results. The algorithm outputs all the detected insect sounds in a new folder named &#8220;dataDir_extf&#8221;, where dataDir is the same as in the previous step. Each sound file is a 1 sec long audio clip originally extracted from the raw recording, with a digital filter applied to remove noise. The occurrence time of each detected sound is saved in a file named &#8220;dataDir_time.mat&#8221;. Observe the example of a detected insect sound in Figure 2.</li>
	</ol>
	</li>
	<li>Detection Algorithm
	<ol>
		<li>Use a 0.1 sec&#160;long sliding window to slide through the recording. The sliding window starts from the beginning of the recording. For each window, follow the steps below.
		<ol>
			<li>Compute the fundamental frequency of the current window.</li>
			<li>If the fundamental frequency is within the range of 100 Hz to 1,200 Hz, then do the following:
			<ol>
				<li>Extract the 1-sec&#160;long audio clip centering at the current window from the recording; apply a digital filter to remove the noise in the clip and save the filtered audio into the folder &#8220;dataDir_extf&#8221;.</li>
				<li>Save the occurrence time of the current window into the file &#8220;dataDir_time&#8221;.</li>
				<li>Move the sliding window to the point that immediately follows the extracted audio clip.</li>
			</ol>
			</li>
			<li>If the fundamental frequency is NOT within the range of 100 Hz to 1,200 Hz, simply move the sliding window 0.01 sec&#160;forward.</li>
		</ol>
		</li>
		<li>Repeat the process until the sliding window reaches the end of the recording.</li>
	</ol>
	</li>
</ol>
4. Insect Classification
<ol>
	<li>Bayesian Classification Using Just the Flying Sound 
	Note: Bayesian classifier is a probabilistic classifier that classifies&#160;an object to its most probable class.
	<ol>
		<li>Sound Feature Computation
		<ol>
			<li>For each insect sound, compute the frequency spectrum of the sound using the Discrete Fourier Transform (DFT). Truncate the frequency spectrum to include only those data points corresponding to the frequency range: 100 Hz to 2,000 Hz. The truncated frequency spectrum is then used in the classification as the &#8220;representative&#8221; of the insect sound. 
			NOTE: The DFT is an algorithm that transforms signals in time domain to the frequency domain. It is a built-in function in most programming libraries, and can be called in the program with just one line of code.</li>
		</ol>
		</li>
		<li>Train a Bayesian classifier
		<ol>
			<li>Use the kNN density estimation approach14 to learn the posterior probability distribution using the sound feature. With the kNN approach, the training phase is to build a training dataset.
			<ol>
				<li>Randomly sample a number of insect sounds from the data collected for each species of insects.</li>
				<li>Follow the steps in Section 4.1.1 and compute the truncated frequency spectrum for each sampled sound. The truncated spectrums together with the samples&#8217; class labels (insect species name) composed the training dataset.</li>
			</ol>
			</li>
		</ol>
		</li>
		<li>Use the Bayesian classifier to classify an unknown insect
		<ol>
			<li>Compute the truncated frequency spectrum of the unknown insect sound.</li>
			<li>Compute the Euclidean distance between the truncated spectrum of the unknown object and all the truncated spectrums in the training dataset.</li>
			<li>Find the top k (k = 8 in this paper) nearest neighbors of the unknown object in the training dataset. Compute the posterior probability of the unknown insect sound belonging to a class as the fraction of the top k nearest neighbors which are labeled as class .</li>
			<li>Classify the unknown object to the class that has the highest posterior probability.</li>
		</ol>
		</li>
	</ol>
	</li>
	<li>Add a Feature to the Classifier: Insect &#160;Circadian Rhythm
	<ol>
		<li>Learn the class-conditioned distributions of the occurrence time of insect sound, that is, the &#160;circadian rhythm for each species of insects.
		<ol>
			<li>Obtain the occurrence time of each sound from the detection results (c.f. Section 3.2).</li>
			<li>For each species, build a histogram of the insect sound occurrence time.</li>
			<li>Normalize the histogram so that the area of the histogram is one. The normalized histogram is the &#160;circadian rhythm of the given species. It tells the probability of observing an insect, of that species, in flight within a certain time period.</li>
		</ol>
		</li>
		<li>Classify an unknown &#8220;insect sound&#8221; by combining the &#8220;insect sound&#8221; and the&#160;circadian rhythm
		<ol>
			<li>Given the specific point in time in which the unknown insect sound occurred, obtain the probability of observing an insect of class based on the&#160;circadian rhythm of class. 
			NOTE: The&#160;circadian rhythm is a probability distribution. It is an array specifying the probability of detecting a sound, produced by a specific species of insects, at a specific time of day. So once a time is given, one can simply check the array to get the probability.</li>
			<li>Follow the steps in section 4.1.2 to compute the posterior probability that the unknown sound belongs to class using the sound features. Multiply the posterior probability to the results from the previous step to get the new posterior probability.</li>
			<li>Classify the &#8220;unknown sound&#8221; to the class that has the highest new posterior probability.</li>
		</ol>
		</li>
	</ol>
	</li>
	<li>Add One More Feature to the Classifier: Insect Geographic Distribution
	<ol>
		<li>Learn the geographic distribution of the species of interest, either from data published in historical records, relevant literature, or simply gather the first-hand knowledge from field technicians/biologists. For demonstration purposes, use a simulation of the graphic distribution, as shown in Figure 7.</li>
		<li>Classify an &#8220;unknown insect sound&#8221; using &#8220;flying sound&#8221; and the two additional features.
		<ol>
			<li>Given the geographic location where the insect sound was intercepted, compute the probability of observing an insect from class at that specific location using the graphic distribution of species.
			<ol>
				<li>Follow steps in section 4.2.2 and compute the posterior probability that the &#8220;unknown sound&#8221; belongs to class using the sound features and the&#160;circadian rhythms. Multiply the outcome of this step to the results from the previous step, in order to get the new posterior probability.</li>
			</ol>
			</li>
			<li>Classify the &#8220;unknown sound&#8221; to the class that has the highest new posterior probability.</li>
		</ol>
		</li>
	</ol>
	</li>
	<li>A General Framework for Adding Features
	<ol>
		<li>Consider the Bayesian classifier that uses just sound features as the primary classifier. Follow the steps below to add new features to the classifier.
		<ol>
			<li>In the training phase, learn the class-conditioned density functions of the new feature.</li>
			<li>In the classification phase, given the new feature of the &#8220;unknown sound&#8221;, compute the probability of observing the feature in class using the density functions learned in the previous step. Multiply the new probability to the previous posterior probability corresponding to the &#8220;unknown sound&#8221; belonging to class which were computed based on just the odd features, to obtain the new posterior probability. Classify the unknown object to the class that has highest new posterior probability.</li>
		</ol>
		</li>
	</ol>
	</li>
</ol>

<ol>
	<li>Kahn, M. C., Celestin, W., Offenhauser, W. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Recording+of+sounds+produced+by+certain+disease-carrying+mosquitoes.">Recording of sounds produced by certain disease-carrying mosquitoes.</a> Science. 101 (2622), 335-336 (1945).</li><li>Reed, S. C., Williams, C. M., Chadwick, L. E. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Frequency+of+wing-beat+as+a+character+for+separating+species+races+and+geographic+varieties+of+Drosophila.">Frequency of wing-beat as a character for separating species races and geographic varieties of Drosophila.</a> Genetics. 27 (3), 349 (1942).</li><li>Belton, P., Costello, R. A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Flight+sounds+of+the+females+of+some+mosquitoes+of+Western+Canada.">Flight sounds of the females of some mosquitoes of Western Canada.</a> Entomologia experimentalis et applicata. 26 (1), 105-114 (1979).</li><li>Mankin, R. W., Machan, R., Field Jones, R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=testing+of+a+prototype+acoustic+device+for+detection+of+Mediterranean+fruit+flies+flying+into+a+trap.">testing of a prototype acoustic device for detection of Mediterranean fruit flies flying into a trap.</a> Proc. 7th Int Symp Fruit Flies of Economic Importance. , 10-15 (2006).</li><li>Raman, D. R., Gerhardt, R. R., Wilkerson, J. B. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Detecting+insect+flight+sounds+in+the+field+Implications+for+acoustical+counting+of+mosquitoes.">Detecting insect flight sounds in the field Implications for acoustical counting of mosquitoes.</a> Transactions of the ASABE. 50 (4), 1481 .</li><li>Moore, A., Miller, J. R., Tabashnik, B. E., Gage, S. H. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Automated+identification+of+flying+insects+by+analysis+of+wingbeat+frequencies.">Automated identification of flying insects by analysis of wingbeat frequencies.</a> Journal of economic entomology. 79 (6), 1703-1706 (1986).</li><li>Moore, A., Miller, R. H. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Automated+identification+of+optically+sensed+aphid+(Homoptera:+Aphidae)+wingbeat+waveforms.">Automated identification of optically sensed aphid (Homoptera: Aphidae) wingbeat waveforms.</a> Annals of the Entomological Society of America. 95 (1), 1-8 (2002).</li><li>Li, Z., Zhou, Z., Shen, Z., Yao, Q. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Automated+identification+of+mosquito+(diptera:+Culicidae)+wingbeat+waveform+by+artificial+neural+network.">Automated identification of mosquito (diptera: Culicidae) wingbeat waveform by artificial neural network.</a> Artificial Intelligence Applications and Innovations. , 483-489 (2005).</li><li>Moore, A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Artificial+neural+network+trained+to+identify+mosquitoes+in+flight.">Artificial neural network trained to identify mosquitoes in flight.</a> Journal of insect behavior. 4 (3), 391-396 (1991).</li><li>Halevy, A., Norvig, P., Pereira, F. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+unreasonable+effectiveness+of+data.">The unreasonable effectiveness of data.</a> Intelligent Systems, IEEE. 24 (2), 8-12 (2009).</li><li>Banko, M., Brill, E. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Mitigating+the+paucity-of-data+problem+exploring+the+effect+of+training+corpus+size+on+classifier+performance+for+natural+language+processing.">Mitigating the paucity-of-data problem exploring the effect of training corpus size on classifier performance for natural language processing.</a> , 1-5 (2001).</li><li>Shotton, J., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Real-time+human+pose+recognition+in+parts+from+single+depth+images.">Real-time human pose recognition in parts from single depth images.</a> Communications of the ACM. 56 (1), 116-124 (2013).</li><li>Ilyas, P. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Classifying+insects+on+the+fly.">Classifying insects on the fly.</a> Ecological Informatics. , (2013).</li><li>Mack, Y. P., Rosenblatt, M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Multivariate+k-nearest+neighbor+density+estimates.">Multivariate k-nearest neighbor density estimates.</a> Journal of Multivariate Analysis. 9 (1), 1-15 (1979).</li><li>Benedict, M., Robinson, A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+first+releases+of+transgenic+mosquitoes+an+argument+for+the+sterile+insect+technique.">The first releases of transgenic mosquitoes an argument for the sterile insect technique.</a> TRENDS in Parasitology. 19 (8), 349-355 (2003).</li><li>. Supporting Materials Available from: <a target="_blank" href="https://sites.google.com/site/insectclassification">https://sites.google.com/site/insectclassification</a></li><li>Chen, Y., Why, A., Batista, G., Mafra-Neto, A., Keogh, E. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Flying+Insect+Classification+with+Inexpensive+Sensors.">Flying Insect Classification with Inexpensive Sensors.</a> Journal of Insect. 27 (5), 657-677 (2014).</li></ol>

The authors declare that they have no competing financial interests.

The sensor/classification framework described here allows the inexpensive and scalable classification of flying insects. The accuracies achievable by the system are good enough to allow the development of commercial products and could be a useful tool in entomological research.
The ability to use inexpensive, noninvasive sensors to accurately and automatically classify flying insects would have significant implications for entomological research. For example, by deploying the system in the fie...

The idea of automatically classifying insects using the incidental sound of their flight dates back to the earliest days of computers and commercially available audio recording equipment1. However, little progress has been made on this problem in the intervening decades. The lack of progress in this pursuit can be attributed to several related factors.
First, the lack of effective sensors has made data collection difficult. Most efforts to collect data have used acoustic microphones2-5. Such devices are extremely sensitive to wind noise and ambient noise in the environment, resulting in very sparse and low-quality data.
Second, compounding these data quality issues is the fact that many researchers have attempted to learn very complicated classification models, especially neural networks6-8. Attempting to learn complicated classification models, with a mere tens of examples, is a recipe for over-fitting.
Third, the difficultly of obtaining data has meant that many researchers have attempted to build classification models with very limited data, as few as 300 instances9 or less. However, it is known that for building accurate classification models, more data is better10-13.
This work addresses all three issues. Optical (rather than acoustic) sensors can be used to record the &#8220;sound&#8221; of insect flight from meters away, with complete invariance to wind noise and ambient sounds. These sensors have allowed the recording of millions of labeled training instances, far more data than all previous efforts combined, and thus help avoid the over-fitting that has plagued previous research efforts. A principled method is shown below that allows the incorporation of additional information into the classification model. This additional information can be as quotidian and as easy-to-obtain as the time-of-day, yet still produce significant gains in accuracy of the model. Finally, it is demonstrated that the enormous amounts of data we collected allow us to take advantage of &#8220;The unreasonable effectiveness of data&#8221;10 to produce simple, accurate and robust classifiers.
In summary, flying insect classification has moved beyond the dubious claims created in the research lab and is now ready for real-world deployment. The sensors and software presented in this work will provide researchers worldwide robust tools to accelerate their research.

<table><tbody><tr><td>Audio recorder: ICD-PX312</td><td>Sony</td><td>4-267-065-11(2)</td><td>With a 8 GB microSD extra memory</td></tr><tr><td>Insectary</td><td>Lee&#039;s Aquarium &amp;amp; Pet Products.</td><td>20088 HerpHaven&amp;reg;, Large Rectangle</td><td>14 1/2&quot; long x 8 3/4&quot; wide x 9 3/4&quot; high. Modified to house insects.</td></tr><tr><td>Laser line generator, 650 nm (red)</td><td>Apinex (www.apinex.com)</td><td>LN60-650</td><td>5 mW. This is a low powered laser, similar to a teacher&#039;s laser pointer</td></tr><tr><td>Photodiode array</td><td>VISHAY SEMICONDUCTOR&amp;nbsp; TEFD4300&amp;nbsp; PIN PHOTODIODE, 650NM, 20DEG, T1</td><td>TEFD4300</td><td>We made a custom array of 15 of these photodiodes wired in parallel.</td></tr><tr><td>Analogue to digital convertor integrated circuit</td><td>Custom made in our lab</td><td></td><td>We made this item ourselves, but an easily available commercial product, Gino PCF8591 AD/DA Converter, provides the same functionality.</td></tr></tbody></table>

flying insect detection and classification with inexpensive sensors

An inexpensive, noninvasive system that could accurately classify flying insects would have important implications for entomological research, and allow for the development of many useful applications in vector and pest control for both medical and agricultural entomology. Given this, the last sixty years have seen many research efforts devoted to this task. To date, however, none of this research has had a lasting impact. In this work, we show that pseudo-acoustic optical sensors can produce superior data; that additional features, both intrinsic and extrinsic to the insect&#8217;s flight behavior, can be exploited to improve insect classification; that a Bayesian classification approach allows to efficiently learn classification models that are very robust to over-fitting, and a general classification framework allows to easily incorporate arbitrary number of features. We demonstrate the findings with large-scale experiments that dwarf all previous works combined, as measured by the number of insects and the number of species considered.

Two experiments are presented here. For both experiments, the data used were randomly sampled from a dataset that contains over 100,000 objects.
The first experiment shows the ability of the proposed classifier to accurately classify different species/sexes of insects. As the classification accuracy depends on the insects to be classified, a single absolute value for classification accuracy will not give the reader a good intuition about the performance of the system. Instead, rather than repo...

Watch this Scientific Journal Video about Flying Insect Detection and Classification with Inexpensive Sensors at JoVE.com

Flying Insect Detection and Classification with Inexpensive Sensors

We proposed a system that uses inexpensive, noninvasive pseudo-acoustic optical sensors to automatically and accurately detect, count, and classify flying insects based on their flying sound.

An inexpensive, noninvasive system that could accurately classify flying insects would have important implications for entomological research, and allow ...

flying-insect-detection-and-classification-with-inexpensive-sensors

Research

JoVE Journal

Bioengineering

25.1K Views.  University of California, Riverside. We proposed a system that uses inexpensive, noninvasive pseudo-acoustic optical sensors to automatically and accurately detect, count, and classify flying insects based on their flying sound.

Video: Flying Insect Detection and Classification with Inexpensive Sensors

Early Metamorphic Insertion Technology for Insect Flight Behavior Monitoring

Early Metamorphosis Insertion Technology (EMIT) is a novel methodology for integrating microfabricated neuromuscular recording and actuation platforms on insects during their metamorphic development. Here, the implants are fused within the structure and function of the neuromuscular system as a result of metamorphic tissue remaking. The implants emerge with the insect where the development of tissue around the electronics during pupal development results in a bioelectrically and biomechanically enhanced tissue interface. This relatively more reliable and stable interface would be beneficial for many researchers exploring the neural basis of the insect locomotion with alleviated traumatic effects caused during adult stage insertions. In this article, we implant our electrodes into the indirect flight muscles of Manduca sexta. Located in the dorsal-thorax, these main flight powering dorsoventral and dorsolongitudinal muscles actuate the wings and supply the mechanical power for up and down strokes. Relative contraction of these two muscle groups has been under investigation to explore how the yaw maneuver is neurophysiologically coordinated. To characterize the flight dynamics, insects are often tethered with wires and their flight is recorded with digital cameras. We also developed a novel way to tether Manduca sexta on a magnetically levitating frame where the insect is connected to a commercially available wireless neural amplifier. This set up can be used to limit the degree of freedom to yawing &#8220;only&#8221; while transmitting the related electromyography signals from dorsoventral and dorsolongitudinal muscle groups.

We present a novel surgical procedure to implant electrodes in Manduca sexta during its early metamorphic stages. This technique allows mechanically stable and electrically reliable coupling with the neuromuscular tissue to study flight neurophysiology dynamics. We also present a novel magnetic levitation platform for tethered studies of insect yaw.

Early Metamorphosis Insertion Technology (EMIT) is a novel methodology for integrating microfabricated neuromuscular recording and actuation platforms on ...

Behavior

Measuring the Flight Ability of the Ambrosia Beetle, Platypus Quercivorus (Murayama), Using a Low-Cost, Small, and Easily Constructed Flight Mill

The ambrosia beetle, Platypus quercivorus (Murayama), is the vector of a fungal pathogen that causes mass mortality of Fagaceae trees (Japanese oak wilt). Therefore, knowing the dispersal capacity may help inform trapping/tree removal efforts to prevent this disease more effectively. In this study, we measured the flight velocity and duration and estimated the flight distance of the beetle using a newly developed flight mill. The flight mill is low cost, small, and constructed using commonly available items. Both the flight mill arm and its vertical axis comprise a thin needle. A beetle specimen is glued to one tip of the arm using instant glue. The other tip is thick due to being covered with plastic, thus it facilitates the detection of rotations of the arm. The revolution of the arm is detected by a photo sensor mounted on an infrared LED, and is indicated by a change in the output voltage when the arm passed above the LED. The photo sensor is connected to a personal computer and the output voltage data are stored at a sampling rate of 1 kHz. By conducting experiments using this flight mill, we found that P. quercivorus can fly at least 27 km. Because our flight mill comprises cheap and small ordinary items, many flight mills can be prepared and used simultaneously in a small laboratory space. This enables experimenters to obtain a sufficient amount of data within a short period.

Measuring the Flight Ability of the Ambrosia Beetle, Platypus Quercivorus Murayama, Using a Low-Cost, Small, and Easily Constructed Flight Mill

We developed a low cost and small flight mill, constructed with commonly available items and easily used in experimentation. Using this apparatus, we measured the flight ability of an ambrosia beetle, Platypus quercivorus.

The ambrosia beetle, Platypus quercivorus (Murayama), is the vector of a fungal pathogen that causes mass mortality of Fagaceae trees (Japanese oak wilt). ...

Environment

A Precise and Autonomous System for the Detection of Insect Emergence Patterns

Existing systems to measure insect emergence patterns have limitations; they are only partially automated and are limited in the maximum number of emerging insects they can detect. In order to obtain precise measurement of insect emergence, it is necessary for systems to be semi-automated and able to measure large numbers of emerging insects. We addressed these issues by designing and building a system that is automated and can measure emergence of up to 1200 insects. We modified the existing &#34;falling-ball&#34; system using Arduino microcontrollers to automate data collection and expand the sample size through multiple data channels. Multiple data channels enable the user to not only increase their sample size, but also allows for multiple treatments to be run simultaneously in a single experiment. Furthermore, we created an R script to automatically visualize the data as a bubble plot, while also calculating the median day and time of emergence. The current system was designed using 3D printing so the user can modify the system to be adjusted for different species of insects. The goal of this protocol is to investigate important questions in chronobiology and stress physiology, using this precise and automated system to measure insect emergence patterns.

Measurement of insect emergence patterns requires precision. Existing systems are only semi-automated and sample size is limited. We addressed these issues by designing a system using microcontrollers to precisely measure the time of emergence of large numbers of emerging insects.

Existing systems to measure insect emergence patterns have limitations; they are only partially automated and are limited in the maximum number of ...

A Wind Tunnel for Odor Mediated Insect Behavioural Assays

Olfaction is the most important sensory mechanism by which many insects interact with their environment and a wind tunnel is an excellent tool to study insect chemical ecology. Insects can locate point sources in a three-dimensional environment through the sensory interaction and sophisticated behavior. The quantification of this behavior is a key element in the development of new tools for pest control and decision support. A wind tunnel with a suitable flight section with laminar air flow, visual cues for in-flight feedback and a variety of options for the application of odors can be used to measure complex behaviour which subsequently may allow the identification of attractive or repellent odors, insect flight characteristics, visual-odor interactions and interactions between attractants and odors lingering as background odors in the environment. A wind tunnel holds the advantage of studying the odor mediated behavioural repertoire of an insect in a laboratory setting. Behavioural measures in a controlled setting provide the link between the insect physiology and field application. A wind tunnel must be a flexible tool and should easily support the changes to setup and hardware to fit different research questions. The major disadvantage to the wind tunnel setup described here, is the clean odor background which necessitates special attention when developing a synthetic volatile blend for field application.

Here, we describe the construction and use of a wind tunnel for odor mediated behavioural assays with insects. The wind tunnel design facilitates the release of odor sources by several methods, with and without visual stimuli. Wind tunnel experiments are important methods to identify behaviorally active volatile chemicals.

Olfaction is the most important sensory mechanism by which many insects interact with their environment and a wind tunnel is an excellent tool to study ...

Low-Cost Automated Flight Intercept Trap for the Temporal Sub-Sampling of Flying Insects Attracted to Artificial Light at Night

Sampling methods are selected depending on the targeted species or the spatial and temporal requirements of the study. However, most methods for passive sampling of flying insects have a poor temporal resolution because it is time-consuming, costly and/or logistically difficult to perform. Effective sampling of flying insects attracted to artificial light at night (ALAN) requires sampling at user-defined time points (nighttime only) across well-replicated sites resulting in major time and labor-intensive survey effort or expensive automated technologies. Described here is a low-cost automated intercept trap that requires no specialist equipment or skills to construct and operate, making it a viable option for studies that require temporal sub-sampling across multiple sites. The trap can be used to address a wide range of other ecological questions that require a greater temporal and spatial scale than is feasible with previous trap technology.

To study the impacts of artificial light at night (ALAN) on nocturnal flying insects, sampling needs to be confined to nighttime. The protocol describes a low-cost automated flight intercept trap that allows researchers to sample at user-defined periods with increased replication.

Sampling methods are selected depending on the targeted species or the spatial and temporal requirements of the study. However, most methods for passive ...

Building an Enhanced Flight Mill for the Study of Tethered Insect Flight

Makerspaces have a high potential of enabling researchers to develop new techniques and to work with novel species in ecological research. This protocol demonstrates how to take advantage of the technology found in makerspaces in order to build a more versatile flight mill for a relatively low cost. Given that this study extracted its&#160;prototype from flight mills built in the last decade, this protocol focuses more on outlining divergences made from the simple, modern flight mill. Previous studies have already shown how advantageous flight mills are to measuring flight parameters such as speed, distance, or periodicity. Such mills have allowed researchers to associate these parameters with morphological, physiological, or genetic factors. In addition to these advantages, this study discusses the benefits of using the technology in makerspaces, like 3D printers and laser cutters, in order to build a more flexible, sturdy, and collapsible flight mill design. Most notably, the 3D printed components of this design allow the user to test insects of various sizes by making the heights of the mill arm and infrared (IR) sensors adjustable. The 3D prints also enable the user to easily disassemble the machine for quick storage or transportation to the field. Moreover, this study makes greater use of magnets and magnetic paint to tether insects with minimal stress. Lastly, this protocol details a versatile analysis of flight data through computer scripts that efficiently separate and analyze differentiable flight trials within a single recording. Although more labor-intensive, applying the tools available in makerspaces and on online 3D modeling programs facilitates multidisciplinary and process-orientated practices and helps researchers avoid costly, premade products with narrowly adjustable dimensions. By taking advantage of the flexibility and reproducibility of technology in makerspaces, this protocol promotes creative flight mill design and inspires open science.

This protocol uses three-dimensional (3D) printers and laser cutters found in makerspaces in order to create a more flexible flight mill design. By using this technology, researchers can reduce costs, enhance design flexibility, and generate reproducible work when constructing their flight mills for tethered insect flight studies.

Makerspaces have a high potential of enabling researchers to develop new techniques and to work with novel species in ecological research. This protocol ...

Engineering

Using Insect Electroantennogram Sensors on Autonomous Robots for Olfactory Searches

Robots designed to track chemical leaks in hazardous industrial facilities1 or explosive traces in landmine fields2 face the same problem as insects foraging for food or searching for mates3: the olfactory search is constrained by the physics of turbulent transport4. The concentration landscape of wind borne odors is discontinuous and consists of sporadically located patches. A pre-requisite to olfactory search is that intermittent odor patches are detected. Because of its high speed and sensitivity5-6, the olfactory organ of insects provides a unique opportunity for detection. Insect antennae have been used in the past to detect not only sex pheromones7 but also chemicals that are relevant to humans, e.g., volatile compounds emanating from cancer cells8 or toxic and illicit substances9-11. We describe here a protocol for using insect antennae on autonomous robots and present a proof of concept for tracking odor plumes to their source. The global response of olfactory neurons is recorded in situ in the form of electroantennograms (EAGs). Our experimental design, based on a whole insect preparation, allows stable recordings within a working day. In comparison, EAGs on excised antennae have a lifetime of 2 hr. A custom hardware/software interface was developed between the EAG electrodes and a robot. The measurement system resolves individual odor patches up to 10 Hz, which exceeds the time scale of artificial chemical sensors12. The efficiency of EAG sensors for olfactory searches is further demonstrated in driving the robot toward a source of pheromone. By using identical olfactory stimuli and sensors as in real animals, our robotic platform provides a direct means for testing biological hypotheses about olfactory coding and search strategies13. It may also prove beneficial for detecting other odorants of interests by combining EAGs from different insect species in a bioelectronic nose configuration14 or using nanostructured gas sensors that mimic insect antennae15.

We describe a protocol for using insect antennae in the form of electroantennograms (EAGs) on autonomous robots. Our experimental design allows stable recordings within a day and resolves individual odor patches up to 10 Hz. The efficiency of EAG sensors for olfactory searches is demonstrated in driving a robot toward an odor source.

Robots designed to track chemical leaks in hazardous industrial facilities1 or explosive traces in landmine fields2 face the same problem as insects ...

Neuroscience

Electroantennography-based Bio-hybrid Odor-detecting Drone using Silkmoth Antennae for Odor Source Localization

Small drones with chemical or biosensor devices that can detect airborne odorant molecules have attracted considerable attention owing to their applicability in environmental and security monitoring and search-and-rescue operations. Small drones with commercial metal-oxide-semiconductor (MOX) gas sensors have been developed for odor source localization; however, their real-time-odor-detection performance has proven inadequate. However, biosensing technologies based on insect olfactory systems exhibit relatively high sensitivity, selectivity, and real-time response with respect to odorant molecules compared to commercial MOX gas sensors. In such devices, excised insect antennae function as portable odorant biosensor elements and have been found to deliver excellent sensing performance. This study presents experimental protocols for odorant-molecule detection in the air using a small autonomous bio-hybrid drone based on a mountable electroantennography (EAG) device incorporating silkmoth antennae.
We developed a mountable EAG device including sensing/processing parts with a Wi-Fi module. The device was equipped with a simple sensor enclosure to enhance the sensor directivity. Thus, odor source localization was conducted using the spiral-surge algorithm, which does not assume an upwind direction. The experimental bio-hybrid odor-detecting drone identified real-time odorant-concentration differences in a pseudo-open environment (outside a wind tunnel) and localized the source. The developed drone and associated system can serve as an efficient odorant molecule-detection tool and a suitable flight platform for developing odor source localization algorithms owing to its high programmability.

This study introduces experimental protocols for a bio-hybrid odor-detecting drone based on silkmoth antennae. The operation of an experimental electroantennogram device with silkmoth antennae is presented, in addition to the structure of a bio-hybrid drone designed for odor source localization using the spiral-surge algorithm.

Small drones with chemical or biosensor devices that can detect airborne odorant molecules have attracted considerable attention owing to their ...

A Simple Flight Mill for the Study of Tethered Flight in Insects

Flight in insects can be long-range migratory flights, intermediate-range dispersal flights, or short-range host-seeking flights. Previous studies have shown that flight mills are valuable tools for the experimental study of insect flight behavior, allowing researchers to examine how factors such as age, host plants, or population source can influence an insects' propensity to disperse. Flight mills allow researchers to measure components of flight such as speed and distance flown. Lack of detailed information about how to build such a device can make their construction appear to be prohibitively complex. We present a simple and relatively inexpensive flight mill for the study of tethered flight in insects. Experimental insects can be tethered with non-toxic adhesives and revolve around an axis by means of a very low friction magnetic bearing. The mill is designed for the study of flight in controlled conditions as it can be used inside an incubator or environmental chamber. The strongest points are the very simple electronic circuitry, the design that allows sixteen insects to fly simultaneously allowing the collection and analysis of a large number of samples in a short time and the potential to use the device in a very limited workspace. This design is extremely flexible, and we have adjusted the mill to accommodate different species of insects of various sizes.

Flight in insects is influenced by a number of factors and the propensity to disperse is an important variable in understanding insect ecology and biological control strategies. We describe the construction and use of a simple, relatively inexpensive, and flexible flight mill for measuring parameters of tethered flight in insects.

Flight in insects can be long-range migratory flights, intermediate-range dispersal flights, or short-range host-seeking flights. Previous studies have ...

Insect-controlled Robot: A Mobile Robot Platform to Evaluate the Odor-tracking Capability of an Insect

Robotic odor source localization has been a challenging area and one to which biological knowledge has been expected to contribute, as finding odor sources is an essential task for organism survival. Insects are well-studied organisms with regard to odor tracking, and their behavioral strategies have been applied to mobile robots for evaluation. This "bottom-up" approach is a fundamental way to develop biomimetic robots; however, the biological analyses and the modeling of behavioral mechanisms are still ongoing. Therefore, it is still unknown how such a biological system actually works as the controller of a robotic platform. To answer this question, we have developed an insect-controlled robot in which a male adult silkmoth (Bombyx mori) drives a robot car in response to odor stimuli; this can be regarded as a prototype of a future insect-mimetic robot. In the cockpit of the robot, a tethered silkmoth walked on an air-supported ball and an optical sensor measured the ball rotations. These rotations were translated into the movement of the two-wheeled robot. The advantage of this "hybrid" approach is that experimenters can manipulate any parameter of the robot, which enables the evaluation of the odor-tracking capability of insects and provides useful suggestions for robotic odor-tracking. Furthermore, these manipulations are non-invasive ways to alter the sensory-motor relationship of a pilot insect and will be a useful technique for understanding adaptive behaviors.

The capability to localize an odor source is necessary for insect survival and is expected to be applicable to artificial odor-tracking. The insect-controlled robot is driven by an actual silkmoth and enables us to evaluate the odor-tracking capability of insects through a robotic platform.

Robotic odor source localization has been a challenging area and one to which biological knowledge has been expected to contribute, as finding odor ...