Sensor Data Fusion for Improved Sense and Avoid
by
Stanley
D. Pebsworth
Embry-Riddle
Aeronautical University
May
2016
A Research Project Submitted to the
Worldwide Campus in partial fulfillment of the requirements for course UNSY 605,
Unmanned Systems Sensing, Perception, and Processing
Abstract
See and Avoid is a Federal Aviation
Administration (FAA) requirement to operate airborne systems within our
National Airspace. With the increased
awareness of the need and want to adapt unmanned systems into the National
Airspace, the need for improved Sense and Avoid technology has surfaced. Current issues are the rate in which unmanned
systems can react and avoid a collision as compared to their human
counterpart. The FAA and the National
Aeronautics and Space Administration (NASA) have been collaborating to
determine a safe way to incorporate unmanned systems into the current
air-traffic system alongside commercial and private manned systems. This research will identify current hardware
and software relevant to sensor fusion and its application towards the See and
Avoid requirements. With the use of
scholarly and peer reviewed material, this paper will review historical flight
testing of Unmanned Aerial Systems (UAS) used to test Sense and Avoid
technology. Alternatively, this research
will identify the underlying issues associated with the human factors in sense
and avoid and relate how technology addresses these factors.
Keywords: unmanned aircraft systems,
sense and avoid, sensor fusion, situational awareness
Sensor Data Fusion for Improved Sense and Avoid
Currently
Part 91.113 of the Federal Aviation Regulations (FAR) outlines the requirements
for see and avoid (GPO, 2016). In
summary, the regulation states that “vigilance shall be maintained by the pilot
of an aircraft to see and avoid other aircraft” and was last amended in August
of 2004 (FAA, 2004). A committee formed
by the FAA is looking to amend Part 91.113 to include the allowance of see and
avoid through electronic means. A UAS
subcommittee met for the first time on July 30, 2013 and is responsible for
outlining the requirements for electronic sense and avoid (Carey, 2013).
NASA
has been working with the FAA to aide in research and outline the technical
challenges associated with the integration of UAS into the national
airspace. NASA has chaired a program
specifically for this project and key stakeholders have been identified to
foster unencumbered national airspace access for civil and commercial UAS
(Hackenberg, 2014). Figure 1 shows the key stakeholders for the UAS integration
subcommittee.
Figure 1. Key stakeholder
for UAS integration. Adapted from “UAS integration in the NAS project,” by D.
Hackenberg, 2014, NASA.
Problem Statement
Currently unmanned aerial
systems (UAS) are restricted from operations within the national airspace. This restriction requires the operator of the
UAS to seek special authorization to operate their system within the national
airspace. As mandated by the 2012 FAA re-authorization act, safe integration of UAS systems started in 2015 (Carey,
2013). The safe integration of UAS into
the national airspace is contingent on implementation of a safe and
comprehensive aviation program (Hottman, Hansen, & Berry, 2009). This program will require the complete understanding
of sense and avoid technology in order to provide solutions that will enhance
safety and provide UAS with access to the national airspace. Therefore, it is incumbent upon key
stakeholders in UAS integration to address not only safety on integration, but
also address the technological requirements that will make up the electronic
means of sense and avoid.
Significance of Problem Statement
After years of development for
military use, UAS have reached a culminating point and are starting to be
applied more and more to civilian and commercial tasks. The tasks proposed for UAS application are:
environmental, emergency, communications, monitoring, as well as commercial
applications in photography, agriculture, chemical application, and
transportation. UAS have the capability
to offer major advantages when applied to these applications. Currently, there are several companies
producing hundreds of UAS designs. Of
course major defense contractors such as Boeing, Lockheed, and BAE are
involved, but there are also new companies emerging to try and grab their share
of the market. The US currently holds
approximately 64% of the UAS market share.
It is predicted that by 2020, the UAS market growth will reach an annual
expenditure of 11.3 billion dollars for research, development and procurement (Angelov,
2012). Figure 2 shows the forecasted
growth for the world UAV market.
Figure 2. World UAV Forecast.
Adapted from “Sense and avoid in UAS:
Research and applications,” by P. P. Angelov, 2012 p.18. Hoboken: John
Wiley & Sons.
The main funding for
research and development for future UAS systems is the US Department of Defense
(DoD). In the Unmanned Systems
Integrated Road-map published by the DoD, it was stated that the performance of
UAS must evolve significantly in order for their safe integration into the
national airspace (Angelov, 2012). As
civil and commercial UAS begin to be applied toward the possible missions
discussed, they will need access to the national airspace.
It was determined in 2007 by
an FAA General Aviation Research board that nearly 54% of the current FAA
Regulations would have to be revised in order to address UAS integration (Dalamagkidis,
Valavanis, & Piegl, 2009). The issue
is that the current regulations have been developed over decades of experience
and this new revision to integrate UAS will have very little experience to draw
from. Regardless, this new integration
will be without problems and will require a complete understanding of the
differences and challenges that may require a different way of thinking. The
intent of this research is to focus primarily on the see and avoid requirement
imposed by the FAA and the possible electronic means for which these
requirements may be met.
Alternative Actions for See and Avoid
Sense
and Avoid is the technology designed to replace the human pilot’s requirement
for See and Avoid (Angelov, 2012). Sense
and avoid technology will be required to avoid hazards such as aircraft,
gliders, balloons and other UAS (Angelov, 2012). There will also be the requirement to avoid
hazardous obstacles such as buildings, towers, power lines and birds. Sense and avoid must be able to provide
detection, tracking, evaluation, prioritization, declaration, maneuver
determination, and maneuver command execution.
There are currently two
primary cooperative technologies that aide in the tasks of detection and
tracking. These systems are “the Traffic
Alert and Collision Avoidance System (TCAS) and the Automatic Dependent
Surveillance Broadcast (ADS-B) system” (Angelov, 2012). The issue with these two systems is that it
requires that other aircraft be equipped with like systems. Non-cooperative technologies that aide in
sense and avoid are radar, laser, optical, and acoustic systems (Angelov,
2012). No single approach provides the
necessary safety level for See and Avoid, therefore the fusion of these
cooperative and non-cooperative technologies is possibly the best alternative.
There
are also human factors to consider depending on the level of UAS autonomy. The first issue is with the removal of the
human from the cockpit the type of feedback perceived by the operator is in
question and how will sensory perception be relayed to the operator. The second issue is with the removal of the
human, he/she is now reduced to simply a monitor of systems and the degradation
of pilot skill may be degraded. Finally,
there is the issue of the transition of operator skill from direct control to
indirect cognitive activity (Angelov, 2012).
Research
in the area of UAS Sense and Avoid technology has found a multitude of
alternative possibilities for see and avoid.
Fasano, Accardo, Tirri, Moccia, & Lellis, (2015), conducted research
and proposed alternative algorithms for an obstacle detection and tracking
system based on the integration of radar and electro optical/infrared
cameras. Their data fusion architecture
was based on a hierarchy of sensors, cross-sensor cueing, and central-level
fusion. Radar is the primary sensor in
their proposal while using electro optical and infrared sensors as auxiliary
sources that improve accuracy (Fasano, et al, 2015). These sensors were adapted to a fixed wing
aircraft (Flying Laboratory for Aeronautical Research (FLARE)) along with
autonomous navigation equipment as depicted in Figure 3.
Figure 3. Sense and avoid
system hardware architecture, and FLARE aircraft. Adapted from “Morphological
filtering and target tracking for vision-based UAS sense and avoid,” by G.
Fasano, D. Accardo, A. E. Tirri, A. Moccia, & E. D. Lellis, 2015.
In research conducted by
Tirri, Fasano, Accardo, & Moccia, (2014), it was proposed that particle
filter algorithms had less limitations than the common Kalman filter
algorithms.
Particle filters are Bayesian estimators
that resolve the state estimation problems by determining the probability
density function (PDF) of an unknown random vector using a weighted sum of
delta functions. These filters have less limitations than the Kalman filter.
Indeed, they can exploit nonlinear process and measurement models and they can
be used with any form of system noise statistical distribution (Tirri, et al
p.4, 2014).
Sensors
are being developed with commercial off the shelf parts such as Frequency
Modulated Continuous Wave (FMCW) radar sensors.
As depicted in Figure 4, these sensors can provide distance and azimuth
of possible targets in a unit as small as 3 x 2 x 1 ½ inch weighing less than 2
ounces (Mackie, Spencer, & Warnick, 2014).
Figure 4. Radar board
with transceiver MMIC and RF signal chains. Adapted from “Compact FMCW radar
for a UAS sense and avoid system,” by J. Mackie, J. Spencer, & K. F.
Warnick, 2014.
Due
to their size and weight, some small UAS would be unable to carry the
additional electronics onboard to meet the requirements for see and avoid. There has been research conducted on ground
based systems that could possibly fill this void. Research by Barott, Coyle, Dabrowski,
Hockley, & Stansbury, (2014), proposed that a passive radar could be used
to monitor an area around the UAS for hazards while a second EO/IR sensor is
used to further classify hazards detected by the radar sensor.
Passive
sensor target tracking has its problems and has been widely reviewed in
literature. Some specific issues are
detection range and the maneuver capability of the airframe to avoid a collision. Estimated ranges and closure rates are often
unstable due to calculations having to be made as both the own-ship and target
aircraft are maneuvering. Kalman
filtering can be used for target state estimation and increased accuracy and
detection and classification ranges.
Once an obstacle is confirmed and a template generated, the target can
be tracked using morphological filtering that produces modified spherical
coordinates that stabilizes the target state estimation (Fasano, Accardo,
Tirri, Moccia, & De Lellis, 2014).
Recommendations for See and Avoid
Sense and Avoid technology
will soon be available and either replace or complement the human pilot’s
requirement for See and Avoid. This new
sense and avoid technology will be required to avoid hazards as well as
obstacles. Sense and avoid must be able to
provide the human operator adequate fidelity to allow for accurate hazard and
obstacle detection, tracking and avoidance.
It is proposed that both cooperative
and non-cooperative technologies to aide in the tasks of detection tracking and
avoidance must be used to fill the requirements of see and avoid though
electronic means. Currently, the best
recommendation is the sensor fusion of TCAS, ADS-B and EO/IR sensors together
with particle filter algorithms as proposed by Tirri, Fasano, Accardo, &
Moccia, (2014). Research has proven that
these fused systems can be made small enough for SWAP UAS.
The issue with this
recommended system is that it requires that other participating aircraft be
equipped with like systems. To combat
this issue, it is recommended that there be requirements in FAA regulation that
define the equipment required for both private and commercial UAS. It is also proposed that any UAS operating
above 400 feet above ground level (AGL) or within and airports defined airport
traffic area (ATA) be required to follow the commercial electronic see and
avoid requirements. Figure 5 outlines
the proposed UAS requirements. See
Appendix A for airspace classification.
Class
G, E airspace
|
|||||
Surface
to 400ft
|
TCAS or
ADS-B
|
EO/IR
Sensors
|
401ft
& above
|
TCAS or
ADS-B
|
EO/IR
Sensors
|
Private
|
NR
|
NR
|
Private
|
REQ
|
REQ
|
Commercial
|
REQ
|
REQ
|
Commercial
|
REQ
|
REQ
|
Class
A, B, C, D airspace
|
|||||
Surface
to 400ft
|
TCAS or
ADS-B
|
EO/IR
Sensors
|
401ft
& above
|
TCAS or
ADS-B
|
EO/IR
Sensors
|
Private
|
REQ
|
NR
|
Private
|
REQ
|
REQ
|
Commercial
|
REQ
|
REQ
|
Commercial
|
REQ
|
REQ
|
Figure
5. Proposed electronic sense and avoid requirements.
No single approach to the
requirements for see and avoid through electronic means will provide the
necessary safety level required however, the fusion of these cooperative and
non-cooperative technologies is possibly the best alternative. It is further recommended that this
possibility of applying this technology to manned aircraft be considered to
further enhance the safety within the national airspace system.
References
Angelov, P. P. (2012). Sense
and avoid in UAS: Research and applications (2nd;1; ed.). Hoboken: John
Wiley & Sons.
Barott, W. C., Coyle, E.,
Dabrowski, T., Hockley, C., & Stansbury, R. S. (2014). Passive multispectral sensor architecture for radar-EOIR sensor fusion
for low SWAP UAS sense and avoid. Paper presented at the 1188-1196.
doi:10.1109/PLANS.2014.6851491
Carey, B. (2013). FAA plans unmanned ‘sense and avoid’ rule in
2016. AINonline. Retrieved from http://www.ainonline.com/aviation-news/air-transport/2013-07-22/faa-plans-unmanned-sense-and-avoid-rule-2016
Dalamagkidis, K.,
Valavanis, K., & Piegl, L. A. (2009). On integrating unmanned aircraft
systems into the national airspace system: Issues, challenges, operational
restrictions, certification, and recommendations. New York: Springer.
Fasano, G., Accardo, D.,
Tirri, A. E., Moccia, A., & De Lellis, E. (2014). Morphological filtering and target tracking for vision-based UAS sense
and avoid. Paper presented at the 430-440. doi:10.1109/ICUAS.2014.6842283
Fasano, G., Accardo, D.,
Tirri, A. E., Moccia, A., & Lellis, E. D. (2015). Sky region obstacle
detection and tracking for vision-based UAS sense and avoid. Journal of
Intelligent & Robotic Systems, doi:10.1007/s10846-015-0285-0
FAA. (2004). Code of Federal Regulations. Retrieved
from http:// rgl.faa.gov/Regulatory_and _Guidance_Library/rgFAR.nsf/0/934f0a02e17e7de086256eeb005192fc!OpenDocument
GPO. (2016). Electronic code of federal regulations.
Retrieved from http://www.ecfr.gov/cgi-bin/text-idx?&c=ecfr&tpl=/ecfrbrowse/Title14/14tab_02.tpl
Hackenberg, D. (2014).
NASA UAS integration in the NAS project. Paper presented at the 1-11.
doi:10.1109/ICNSurv.2014.6820068
Hottman, S.B., Hansen,
K.R., Berry, M. (2009). Literature review
on detect, sense, and avoid technology for unmanned aircraft systems.
Retrieved from http://www.tc.faa.gov/its /worldpac/techrpt/ar0841.pdf
Mackie, J., Spencer, J.,
& Warnick, K. F. (2014). Compact FMCW
radar for a UAS sense and avoid system. Paper presented at the 989-990.
doi:10.1109/APS.2014.6904822
Tirri, A. E., Fasano, G.,
Accardo, D., & Moccia, A. (2014). Particle filtering for obstacle tracking
in UAS sense and avoid applications. The Scientific World Journal, 2014,
280478. doi:10.1155/2014/280478
Appendix A
Figure
1. Airspace Classification. Adapted from “The National Airspace System”
by http://www.americanflyers.net
