Sensor Data Fusion for Improved Sense and Avoid

 

Sensor Data Fusion for Improved Sense and Avoid

by

Stanley D. Pebsworth

Embry-Riddle Aeronautical University

May 2016

A Research Project Submitted to the Worldwide Campus in partial fulfillment of the requirements for course UNSY 605, Unmanned Systems Sensing, Perception, and Processing

Abstract

See and Avoid is a Federal Aviation Administration (FAA) requirement to operate airborne systems within our National Airspace.  With the increased awareness of the need and want to adapt unmanned systems into the National Airspace, the need for improved Sense and Avoid technology has surfaced.  Current issues are the rate in which unmanned systems can react and avoid a collision as compared to their human counterpart.  The FAA and the National Aeronautics and Space Administration (NASA) have been collaborating to determine a safe way to incorporate unmanned systems into the current air-traffic system alongside commercial and private manned systems.  This research will identify current hardware and software relevant to sensor fusion and its application towards the See and Avoid requirements.  With the use of scholarly and peer reviewed material, this paper will review historical flight testing of Unmanned Aerial Systems (UAS) used to test Sense and Avoid technology.  Alternatively, this research will identify the underlying issues associated with the human factors in sense and avoid and relate how technology addresses these factors.

Keywords: unmanned aircraft systems, sense and avoid, sensor fusion, situational awareness

Sensor Data Fusion for Improved Sense and Avoid

            Currently Part 91.113 of the Federal Aviation Regulations (FAR) outlines the requirements for see and avoid (GPO, 2016).  In summary, the regulation states that “vigilance shall be maintained by the pilot of an aircraft to see and avoid other aircraft” and was last amended in August of 2004 (FAA, 2004).  A committee formed by the FAA is looking to amend Part 91.113 to include the allowance of see and avoid through electronic means.  A UAS subcommittee met for the first time on July 30, 2013 and is responsible for outlining the requirements for electronic sense and avoid (Carey, 2013).

            NASA has been working with the FAA to aide in research and outline the technical challenges associated with the integration of UAS into the national airspace.  NASA has chaired a program specifically for this project and key stakeholders have been identified to foster unencumbered national airspace access for civil and commercial UAS (Hackenberg, 2014). Figure 1 shows the key stakeholders for the UAS integration subcommittee.

Figure 1. Key stakeholder for UAS integration. Adapted from “UAS integration in the NAS project,” by D. Hackenberg, 2014, NASA.

Problem Statement

Currently unmanned aerial systems (UAS) are restricted from operations within the national airspace.  This restriction requires the operator of the UAS to seek special authorization to operate their system within the national airspace.  As mandated by the 2012 FAA re-authorization act, safe integration of UAS systems started in 2015 (Carey, 2013).  The safe integration of UAS into the national airspace is contingent on implementation of a safe and comprehensive aviation program (Hottman, Hansen, & Berry, 2009).  This program will require the complete understanding of sense and avoid technology in order to provide solutions that will enhance safety and provide UAS with access to the national airspace.  Therefore, it is incumbent upon key stakeholders in UAS integration to address not only safety on integration, but also address the technological requirements that will make up the electronic means of sense and avoid.

Significance of Problem Statement

            After years of development for military use, UAS have reached a culminating point and are starting to be applied more and more to civilian and commercial tasks.  The tasks proposed for UAS application are: environmental, emergency, communications, monitoring, as well as commercial applications in photography, agriculture, chemical application, and transportation.  UAS have the capability to offer major advantages when applied to these applications.  Currently, there are several companies producing hundreds of UAS designs.  Of course major defense contractors such as Boeing, Lockheed, and BAE are involved, but there are also new companies emerging to try and grab their share of the market.  The US currently holds approximately 64% of the UAS market share.  It is predicted that by 2020, the UAS market growth will reach an annual expenditure of 11.3 billion dollars for research, development and procurement (Angelov, 2012).  Figure 2 shows the forecasted growth for the world UAV market.

Figure 2. World UAV Forecast. Adapted from “Sense and avoid in UAS: Research and applications,” by P. P. Angelov, 2012 p.18. Hoboken: John Wiley & Sons.

The main funding for research and development for future UAS systems is the US Department of Defense (DoD).  In the Unmanned Systems Integrated Road-map published by the DoD, it was stated that the performance of UAS must evolve significantly in order for their safe integration into the national airspace (Angelov, 2012).  As civil and commercial UAS begin to be applied toward the possible missions discussed, they will need access to the national airspace. 

It was determined in 2007 by an FAA General Aviation Research board that nearly 54% of the current FAA Regulations would have to be revised in order to address UAS integration (Dalamagkidis, Valavanis, & Piegl, 2009).  The issue is that the current regulations have been developed over decades of experience and this new revision to integrate UAS will have very little experience to draw from.  Regardless, this new integration will be without problems and will require a complete understanding of the differences and challenges that may require a different way of thinking. The intent of this research is to focus primarily on the see and avoid requirement imposed by the FAA and the possible electronic means for which these requirements may be met.

Alternative Actions for See and Avoid

            Sense and Avoid is the technology designed to replace the human pilot’s requirement for See and Avoid (Angelov, 2012).  Sense and avoid technology will be required to avoid hazards such as aircraft, gliders, balloons and other UAS (Angelov, 2012).  There will also be the requirement to avoid hazardous obstacles such as buildings, towers, power lines and birds.  Sense and avoid must be able to provide detection, tracking, evaluation, prioritization, declaration, maneuver determination, and maneuver command execution. 

There are currently two primary cooperative technologies that aide in the tasks of detection and tracking.  These systems are “the Traffic Alert and Collision Avoidance System (TCAS) and the Automatic Dependent Surveillance Broadcast (ADS-B) system” (Angelov, 2012).  The issue with these two systems is that it requires that other aircraft be equipped with like systems.  Non-cooperative technologies that aide in sense and avoid are radar, laser, optical, and acoustic systems (Angelov, 2012).  No single approach provides the necessary safety level for See and Avoid, therefore the fusion of these cooperative and non-cooperative technologies is possibly the best alternative.

            There are also human factors to consider depending on the level of UAS autonomy.  The first issue is with the removal of the human from the cockpit the type of feedback perceived by the operator is in question and how will sensory perception be relayed to the operator.  The second issue is with the removal of the human, he/she is now reduced to simply a monitor of systems and the degradation of pilot skill may be degraded.  Finally, there is the issue of the transition of operator skill from direct control to indirect cognitive activity (Angelov, 2012).

            Research in the area of UAS Sense and Avoid technology has found a multitude of alternative possibilities for see and avoid.  Fasano, Accardo, Tirri, Moccia, & Lellis, (2015), conducted research and proposed alternative algorithms for an obstacle detection and tracking system based on the integration of radar and electro optical/infrared cameras.  Their data fusion architecture was based on a hierarchy of sensors, cross-sensor cueing, and central-level fusion.  Radar is the primary sensor in their proposal while using electro optical and infrared sensors as auxiliary sources that improve accuracy (Fasano, et al, 2015).  These sensors were adapted to a fixed wing aircraft (Flying Laboratory for Aeronautical Research (FLARE)) along with autonomous navigation equipment as depicted in Figure 3.

Figure 3. Sense and avoid system hardware architecture, and FLARE aircraft. Adapted from “Morphological filtering and target tracking for vision-based UAS sense and avoid,” by G. Fasano, D. Accardo, A. E. Tirri, A. Moccia, & E. D. Lellis, 2015.

In research conducted by Tirri, Fasano, Accardo, & Moccia, (2014), it was proposed that particle filter algorithms had less limitations than the common Kalman filter algorithms.

Particle filters are Bayesian estimators that resolve the state estimation problems by determining the probability density function (PDF) of an unknown random vector using a weighted sum of delta functions. These filters have less limitations than the Kalman filter. Indeed, they can exploit nonlinear process and measurement models and they can be used with any form of system noise statistical distribution (Tirri, et al p.4, 2014).

            Sensors are being developed with commercial off the shelf parts such as Frequency Modulated Continuous Wave (FMCW) radar sensors.  As depicted in Figure 4, these sensors can provide distance and azimuth of possible targets in a unit as small as 3 x 2 x 1 ½ inch weighing less than 2 ounces (Mackie, Spencer, & Warnick, 2014).

Figure 4. Radar board with transceiver MMIC and RF signal chains. Adapted from “Compact FMCW radar for a UAS sense and avoid system,” by J. Mackie, J. Spencer, & K. F. Warnick, 2014.

           

            Due to their size and weight, some small UAS would be unable to carry the additional electronics onboard to meet the requirements for see and avoid.  There has been research conducted on ground based systems that could possibly fill this void.  Research by Barott, Coyle, Dabrowski, Hockley, & Stansbury, (2014), proposed that a passive radar could be used to monitor an area around the UAS for hazards while a second EO/IR sensor is used to further classify hazards detected by the radar sensor.

            Passive sensor target tracking has its problems and has been widely reviewed in literature.  Some specific issues are detection range and the maneuver capability of the airframe to avoid a collision.  Estimated ranges and closure rates are often unstable due to calculations having to be made as both the own-ship and target aircraft are maneuvering.  Kalman filtering can be used for target state estimation and increased accuracy and detection and classification ranges.  Once an obstacle is confirmed and a template generated, the target can be tracked using morphological filtering that produces modified spherical coordinates that stabilizes the target state estimation (Fasano, Accardo, Tirri, Moccia, & De Lellis, 2014).

Recommendations for See and Avoid

Sense and Avoid technology will soon be available and either replace or complement the human pilot’s requirement for See and Avoid.  This new sense and avoid technology will be required to avoid hazards as well as obstacles.  Sense and avoid must be able to provide the human operator adequate fidelity to allow for accurate hazard and obstacle detection, tracking and avoidance.   

It is proposed that both cooperative and non-cooperative technologies to aide in the tasks of detection tracking and avoidance must be used to fill the requirements of see and avoid though electronic means.  Currently, the best recommendation is the sensor fusion of TCAS, ADS-B and EO/IR sensors together with particle filter algorithms as proposed by Tirri, Fasano, Accardo, & Moccia, (2014).  Research has proven that these fused systems can be made small enough for SWAP UAS. 

The issue with this recommended system is that it requires that other participating aircraft be equipped with like systems.  To combat this issue, it is recommended that there be requirements in FAA regulation that define the equipment required for both private and commercial UAS.  It is also proposed that any UAS operating above 400 feet above ground level (AGL) or within and airports defined airport traffic area (ATA) be required to follow the commercial electronic see and avoid requirements.  Figure 5 outlines the proposed UAS requirements.  See Appendix A for airspace classification.

Class G, E airspace
Surface to 400ft	TCAS or ADS-B	EO/IR Sensors	401ft & above	TCAS or ADS-B	EO/IR Sensors
Private	NR	NR	Private	REQ	REQ
Commercial	REQ	REQ	Commercial	REQ	REQ
Class A, B, C, D airspace
Surface to 400ft	TCAS or ADS-B	EO/IR Sensors	401ft & above	TCAS or ADS-B	EO/IR Sensors
Private	REQ	NR	Private	REQ	REQ
Commercial	REQ	REQ	Commercial	REQ	REQ

Figure 5. Proposed electronic sense and avoid requirements.

No single approach to the requirements for see and avoid through electronic means will provide the necessary safety level required however, the fusion of these cooperative and non-cooperative technologies is possibly the best alternative.  It is further recommended that this possibility of applying this technology to manned aircraft be considered to further enhance the safety within the national airspace system.

References

Angelov, P. P. (2012). Sense and avoid in UAS: Research and applications (2nd;1; ed.). Hoboken: John Wiley & Sons.

Barott, W. C., Coyle, E., Dabrowski, T., Hockley, C., & Stansbury, R. S. (2014). Passive multispectral sensor architecture for radar-EOIR sensor fusion for low SWAP UAS sense and avoid. Paper presented at the 1188-1196. doi:10.1109/PLANS.2014.6851491

Carey, B. (2013). FAA plans unmanned ‘sense and avoid’ rule in 2016. AINonline. Retrieved from http://www.ainonline.com/aviation-news/air-transport/2013-07-22/faa-plans-unmanned-sense-and-avoid-rule-2016

Dalamagkidis, K., Valavanis, K., & Piegl, L. A. (2009). On integrating unmanned aircraft systems into the national airspace system: Issues, challenges, operational restrictions, certification, and recommendations. New York: Springer.

Fasano, G., Accardo, D., Tirri, A. E., Moccia, A., & De Lellis, E. (2014). Morphological filtering and target tracking for vision-based UAS sense and avoid. Paper presented at the 430-440. doi:10.1109/ICUAS.2014.6842283

Fasano, G., Accardo, D., Tirri, A. E., Moccia, A., & Lellis, E. D. (2015). Sky region obstacle detection and tracking for vision-based UAS sense and avoid. Journal of Intelligent & Robotic Systems, doi:10.1007/s10846-015-0285-0

FAA. (2004). Code of Federal Regulations. Retrieved from http:// rgl.faa.gov/Regulatory_and _Guidance_Library/rgFAR.nsf/0/934f0a02e17e7de086256eeb005192fc!OpenDocument

GPO. (2016). Electronic code of federal regulations. Retrieved from http://www.ecfr.gov/cgi-bin/text-idx?&c=ecfr&tpl=/ecfrbrowse/Title14/14tab_02.tpl

Hackenberg, D. (2014). NASA UAS integration in the NAS project. Paper presented at the 1-11. doi:10.1109/ICNSurv.2014.6820068

Hottman, S.B., Hansen, K.R., Berry, M. (2009). Literature review on detect, sense, and avoid technology for unmanned aircraft systems. Retrieved from http://www.tc.faa.gov/its /worldpac/techrpt/ar0841.pdf

Mackie, J., Spencer, J., & Warnick, K. F. (2014). Compact FMCW radar for a UAS sense and avoid system. Paper presented at the 989-990. doi:10.1109/APS.2014.6904822

Tirri, A. E., Fasano, G., Accardo, D., & Moccia, A. (2014). Particle filtering for obstacle tracking in UAS sense and avoid applications. The Scientific World Journal, 2014, 280478. doi:10.1155/2014/280478

Appendix A

Figure 1. Airspace Classification. Adapted from “The National Airspace System” by http://www.americanflyers.net