paper title (use style: paper title)edge.rit.edu/edge/p19241/public/detailed design...

Autonomous People Mover (APM) Phase VIRochester Institute of Technology

Multidisciplinary Senior Design (P19241) Benjamin Bruns

Industrial & Systems EngineeringRochester Institute of Technology

David FengComputer Engineering

Rochester Institute of Technology

Andrew BivianoElectrical Engineering


Robert SempleComputer Engineering


Isaac WitlinElectrical Engineering


Ali BataynehComputer Engineering


Connor BillingsElectrical Engineering


I. ABSTRACT

This paper details the sixth phase of the autonomous people mover (APM) project. This multidisciplinary senior design project involves modifying a golf cart so that it can autonomously drive itself across the Rochester Institute of Technology (RIT) campus. The team used the work of the previous phases as a foundation to improve on. Localization, path planning, and obstacle detection are the three main areas of improvement for this phase. Localization improvements allow the cart to utilize a pre-loaded map to accurately determine its current position on campus. The cart then is able to use sensor information to plan a path to another point on campus. This path avoids any obstacles that the APM may encounter. These obstacles can be static or dynamic. Static obstacles include buildings and other obstacles that always exist and are in a fixed location. Dynamic obstacles are obstacles that can move around or aren’t always in the same place such as pedestrians and construction zones. Critical safety upgrades were also added to the APM, making the project safer for future phases to work on.

II. BACKGROUND & PROJECT MOTIVATION

Forecasters predict that autonomous vehicles will take over the automobile market the mid-2020’s. Full autonomy of vehicles is expected to emerge from the already available cruise control, driver assist, and highway autopilot [1]. These autonomous vehicles will begin to make our roadways much safer, our roads less congested, and our lives more efficient overall. According to IHS (the world's top automotive forecaster), for every 10% of cars converted to autonomy, the US economy will save $40 Billion per year [2]. Autonomy in vehicles is at the forefront of technological innovation and becoming a large research area.

The Autonomous People Mover (APM) is a research platform that explores and tests the vast array of technical ways to autonomously drive. The APM combines several high powered sensors and sophisticated GPS technology to navigate across RIT’s campus autonomously. The purpose of the APM is twofold: to provide accessibility to campus goers, and to act as a platform for research and refinement of AV technologies. The APM is in its sixth phase of senior design work and has make great strides over the years. This phases’ main objectives and customer requirements were:

1. Safety of the campus goers, team, and others is the utmost importance and main priority of the team.

2. The APM should be functional for 2019 Imagine RIT and able to successfully demonstrate autonomous capabilities.

3. Add localization so that the APM could know where it was (and where its going) on a static map that was loaded in.

4. The APM should be easy to use and have an intuitive user interface.

5. The APM should use applicable sensors to correctly identify simple objects using machine learning.

The above requirements resulted in the teams engineering requirements in Table 1:

TABLE I. SUMMERIZED ENGINEERING REQUIREMENTS

Summarized Engineering RequirementsDescription Rank Unit

APM Braking Distance 9 <5 feet

Object Detection Distance 9 >10 feetLocalization Accuracy (GPS

Accuracy) 9 >2 feet

LiDAR-based navigation will need to used to avoid obstacles

within certain distance9 >3 feet

GPS-based localization will be used to identify APM location

and heading on static map9 Pass/Fail

A Graphical User Interface will be needed to allow user to

specify cart mode and destination

6 Pass/Fail

A static map will need to be integrated into the cart software

(Rviz)9 Pass/Fail

The static map used to localize and navigate on will need to be

annotated with safezones9 Pass/Fail

Overall the team had many customer requirements but throughout the year the team narrowed the scope of the project due to various constraints. Time was a major constraint for the team. The team only had two semesters to work on the project and even smaller amount of time as the

Rochester Institute of Technology – Multidisciplinary Senior Design – P19241 1

weather greatly impacts the team's ability to drive the cart outside. The team’s budget was constrained to the MSD budget of $500 and a research budget through our main stakeholder and customer, Dr. Raymond Ptucha. The last major constraint for the project team was technical experience and expertise in the project discipline.

III. DESIGN

A. Performance & Handling (Braking, Steering, & Throttle)The braking and throttle have remained relatively the

same since they were added to the APM during Phase II. The motor controller was replaced due to component failure. The new motor controller has the ability to re-implement many of the features that were implemented in either custom hardware or software by APM phase II. This creates for a much cleaner, safer design in the long run. For simplicity, these new capabilities were not utilized and the motor controller was replaced to function just as the old one functioned.

Braking functionality is achieved as follows: a PWM signal is transmitted from an Arduino DUE with three specific duty cycles correlated with moving the actuator forward, backward, and holding steady. Feedback from a potentiometer on the braking actuator is sent to the onboard Arduino DUE that determines whether the actuator should be moving and in which direction. The braking actuator is situated just near the braking pedal on the APM and mechanically pulls the brake pedal down when a braking action is required. Tests were performed in the early stages of development to determine the effectiveness of the braking action. It was found that even at low speed the stopping distance the breaking actuator can achieve is limited. The braking function is described in Figure 1.

Fig. 1. Braking command tree

Since APM Phase V, the power/steering module had failed through overuse. The previous Computer Engineering team implemented a new pulse-width modulated (PWM) motor controller, and edited the APM’s onboard Arduino DUE. Using the ROS library for “ackermann_msgs”, we get a steering value in radians that we convert to a positive and negative representation in degrees. The positive values steer the APM towards the left of the driver, and the negative values steer the APM towards the right of the driver. The incoming radian values from the ackermann are translated to

allow for variable steering, instead of only containing 3 turn angles (a hard left, a hard right, and straight). Once the final, desired value is found, it is published to the arduino, where the hardware takes it and converts it to values that are meaningful to the hardware and mechanical parts. Using a PWM signal to enable or disable the steering, the arduino converts the positive/negative degree value from the tt_core automation process to a specific position and pushes the potentiometer for the steering to the desired position. This position is altered as often as the steering value changes through the automated driving process.

Fig. 2. Steering command tree

B. Navigation & Path PlanningThe APM includes a GPS that is used to localize the

APM and plan paths. The program contains a global planner and a local planner that each read from a global cost map and a local cost map. The global cost map handles static obstacles that do not move over time. An example of such an obstacle would be the buildings on campus. The local planner modifies the route to accommodate obstacles that are not managed by the global planner. These obstacles can include pedestrians and construction barriers. When the APM starts moving, it incorporates input on unexpected obstacles from the sensors to create a new local cost map. This local cost map superimposes itself on top of the existing global cost map, creating a full cost map that the APM uses to navigate the campus. If the APM encounters an obstacle that is not accounted for in the global cost map, the local planner updates the local cost map and creates a new path around the obstacle.

C. Graphical User Interface (GUI)The Graphical User Interface (GUI) of the APM utilizes

the GUI of the previous phase 5 with an addition to some minor enhancements for the current phase 6 (see Figure 3). The original GUI handed off at the beginning of phase 6 provided foundational functionality. The GUI provides essentially a control panel for the user where a map is displayed and a destination may be set for the course of the APM. The GUI also allows for the user to switch between autonomous and manual modes while displaying relevant data such as latitude, longitude, yaw, and position on the GUI


map in near real time. The original manner in which the GUI displayed the APM position on the map only represented position and not yaw of the APM. The minor modification made to the GUI allowed the APM icon to be represented not only by position but also by yaw on the GUI, this allows for the location and orientation of the APM icon to always be seen on the GUI map. One more simple addition to the GUI is the option to switch to predetermined destination points in the backend. The original GUI allows user to set any location within the map as a valid destination whereas with the new option of using predetermined destinations, users may only select from a subset of destinations if activated in the backend.

Fig. 3. Graphical User Interface

D. LocalizationPreviously the APM utilized SLAM to do its localization

and mapping. The SLAM algorithm known as Octomap was used to develop a map and localize on that map. The APM consists of two LiDARs that provided point clouds to Octomap to develop the map and localize. The APM also utilized an onboard GPS and IMU to assist with the the mapping and localization. The GPS and IMU are the VN-200 made by Vectornav. Using SLAM was found to cause problems as the map was always being updated and the APM was unable to localize on a premade map. Therefore, a new map had to be developed for every operation of the APM. Using SLAM also requires large amounts of CPU power and therefore the map making was limited by the memory and capabilities of the onboard CPU. The solution implemented was to utilize the high accuracy capabilities of the VN-200 to do localization on a static map.

In order to implement a pure GPS based localization a static map must be provided to the system. The static map was an image taken from Google maps of the Rochester Institute of Technology’s Destler quad shown below in figure 4. The longitude and latitude of the four corners of the image where document to populate the remaining pixels of the image with longitude and latitudes. Some image processing had to be done to the map so that the teb_local_planner provide a safe and efficient path to the desired destination. The functionality of the Robot Operating System (ROS) requires the GPS coordinates to be converted into ROS frame coordinates. A program was written using Python to take the GPS coordinates from the Vectornav that will continually take the difference between the origin of the map and the location of the APM. This allows for a robust GPS based

localization system that continually tracks the location of the APM on a global static map.

Fig. 4. Non-Annotated Destler Quad Map

E. SafetyDue to the complexity of the APM, in hardware,

software, and mechanically, the team takes safety very serious. The consequences are severe if the team does not make every decision with safety in mind. Our team took large strides to improve the overall safety of the cart, for campus goers, the current team, as well as future teams. A fire extinguisher was purchased to be with the cart at all times. RIT FMS also inspected the cart and clean corrosion off of the batteries, as well as replace some connections. Both the battery receptacle and the charger were replaced as wear and tear of the battery receptacle over the years left it in a state that was a fire hazard. Furthermore, all system tests for the APM are done with safety in mind.

IV. TECHNICAL FEASIBILITY

This phase of the Autonomous People Mover has gone through a myriad of concept and design changes, primarily focused around improving the localization of the cart. Due to the array of available sensors on the golf cart, the team had many options to consider. Upon previous team project hand-off, preliminary testing revealed a large gap between where the cart was and where the team needed it to be to demonstrate at Imagine RIT and meet customer requirements.

A. Performance & Handling (Braking, Steering, & Throttle)As safety is the top priority of the team, our first

feasibility tests were done on the APM brakes. Figure 5 depicts the initial state of the brakes. The APM was rolled for an average of 14.3 feet after the emergency stop was applied.


Fig. 5. Preliminary Emergency Braking Tests

This was unacceptable and our design involved several brake updates. After the changes discussed in the Design section, part A, the team improved the brakes to a point that the team feels is safe.

B. Navigation & Path PlanningA majority of this section of the project was handed off to

P19241 from previous phases of the project. The navigation and path planning was tested every time the cart was taken out. To test this, the team would load in a map to localize on and pick a point on the map (or 2D nav goal). If the subsystem was working correctly, the teb_local_planner (navigation code) would create a safe, efficient, path on the map. Fortunately, the team faced very little issues with this pre-established subsystem.

C. Graphical User Interface (GUI)The testing of the GUI was done in parallel with the

localization of the APM. The cart was simply driven around to known locations and the team checked the GUI to see if the locations, as well as heading, matched up.

D. LocalizationThe localization of the APM was a very high priority for

the team. The testing of this system was important and an iterative process. After loading in annotated maps, the cart was driven around for around a minute for the GPS to collect data. The cart launch file (tt_start) was then ran. By looking at Rviz and the carts surroudings, it was obvious if the cart was successfully localized on the map. The larger issues came from the APM yaw measurement. The heading of the cart was often not correct meaning that the cart on the map had a different heading than it actually had. This caused obvious issues with both localization over time and navigation. Several methods were used to fix this. The team tried starting the cart in a canned initial position as well as many other methods. In the end the team realized that the GPS’s true north was actually true east, so the team simply adjusted degrees radian by 90° or π/2. The APM heading is now correct, however a more robust solution is needed long term.

A large issue found with the APM’s sensors was the lack of consistency in the IMU’s ability to get a reliable and consistent initial yaw. To adjust for the lack of consistency in the initial yaw of the APM, the initial yaw was set to always start up at zero degrees. To the IMU the zero degree point is pointing east. Therefore, the starting position of the APM is

marked to ensure the greatest accuracy of the initial yaw. The location of startup was selected based on a landmark position that was within the path for the showcase of the APM’s performance. The position was also marked using a box that outlined the APM’s footprint to ensure robust and quick localization. Several tests where ran to ensure that the startup location was correct and easily repeatable.

Fig. 6. Example of APM localization on static map in RViz

E. Image SegmentationAs the monocular camera on top of the APM takes in

images, software converts those images into annotated RGB visuals (as seen in Figures 7 and 8). These annotated images help the APM classify pathways (sidewalks, roads, none). The red shows areas the cart cannot travel, the blue is unknown to the cart, and the green is areas the cart can travel. To test the image segmentation the team simply drove the APM throughout campus, seeking out different pathway types, shadows, etc. Figure 7 shows the system working perfectly. As you can see, there are nice clean lines that matchup perfectly with the road edges.

Fig. 7. Example of image segmentation working properly

Figure 8 shows trouble areas where shadows turn into bright sunlight on the concrete. This is a known problem and will be something that needs to be improved moving forward with future project teams.


Fig. 8. Example of shadows affecting image segmentation

Figure 9 shows another trouble area for the APM image segmentation. The cart struggles with pavement type changes. In this example, the pavement changes from asphalt to brick and the cart stops moving (in autonomous mode).

Fig. 9. Example of pavement types changing affecting image segmentation

RESULTS & CONCLUSION

A. Recommended Future Work Properly Mount Braking Motor Controller Select new braking actuator to utilize features of

the new motor controller to increase the speed and variability the actuator to effectively apply braking power. (This would mean the team can move away from Arduino DUE code developed in APM Phase II for a cleaner implementation directly into ROS).

Build URDF file of the cart and its sensors to properly simulate cart physics.

Reduce oversteering Create a robust way to prevent the cart from trying

to drive backwards. This causes the cart to cut the throttle during autonomous driving.

Load in larger annotated maps of RIT campus Increase accuracy of APM yaw (heading) Continue to tune the localization of the APM Collect and add in more trained data sets of campus

locations (for safezone)

B. ConclusionThe team was able to make several localization,

navigation, obstacle detection, and safety improvements to the APM. As of the current (sixth) phase of this project, the APM is able to load a annotated static map and use this map to localize itself using GPS methods. The APM is then able to use the static map to navigate a path within the Destler-Johnson Quad located on RIT’s campus. A dynamic “cost” map created from various sensors allows for the APM to detect suddenly appearing obstacles and react to them by adjusting the path it uses. These improvements move the APM closer to its envisioned end goal, in which it will be able to automatically transport someone to and from any point on the RIT campus with little to no input from the user.

ACKNOWLEDGEMENTS

Thank you to Dr. Alexander Loui our guide, our customer Dr. Ray Ptucha, Clark Hochgraf and members from previous phases Alex Avery and Rob Relyea. Also a big thank you to

the previous CE Senior Design team: Ben Glasstone, Zachariah Carmichael, Frank Cwitkowitz, & Kenneth

Alexopoulos.

REFERENCES

THE BELOW REFERENCES ARE NOT CORRECT

[1] IHS Automotive, “Emerging Technologies: Autonomous Cars- Not If, But When,” IHS Automotive study, : http://press.ihs.com/press-release/automotive/self-driving-cars-moving-industrys-drivers-seat, Jan 2, 2014.

[2] 2nd Annual Willaim P. Eno Paper, “Preparing a Nation for Autonomous Vehicles”, 2013.

[3]


paper title (use style: paper title)edge.rit.edu/edge/p19241/public/detailed design...

Documents