Explore the ways Shield AI can enhance your operations
- Explore Autonomy Development
- Your Platform, Our Autonomy
- Aircraft and Sensors
- Reach Out to Our Team
How Artificial Intelligence Manifests Through Exploration
A conversation with Professor Nathan Michael, Shield AI’s Chief Technology Officer. This is a continuation of our conversation about Trust and Robotic Systems.
In reference to robotic systems, what do we mean by exploration?
Typically, when we talk about exploration with an autonomous robotic system for the types of scenarios that we consider, we’re talking about deploying that robotic system into environments that are unknown. The goal of this deployment and exploration is to enable the individual robot to move through that environment, acquire information, and reduce uncertainty about the environment as it goes.
How is exploration formulated? Said another way, how can we see intelligence manifest through exploration?
If we go back to the definition of exploration as a robot making decisions of where it should go in an environment in order to learn more and reduce uncertainty regarding that environment, then exploration is formulated as a question of figuring out what the current knowledge of the environment is and using that information to determine where the system should go to be able to learn more.
How does this relate to reinforcement learning?
We’ve discussed reinforcement learning as this concept of learning based off some kind of reward. In this case, the reward can be thought of as the additional information gained through exploration.
Can you elaborate on this concept of a “greedy optimization”?
You can think of reinforcement learning as building up experience. The more a system explores a building, the better the understanding it builds of how to explore effectively.
Why is it better to evaluate an optimization problem over a longer time horizon?
When you plan your route to a destination, you don’t think about what the next three steps will be. Instead, you think about where you’re going, and the best trajectory to get from where you are to where you want to go. You’re thinking on a longer-term horizon, and trying to optimize with respect to that, rather than just taking three or four steps and then taking three or four more steps.