Hot Chips 2021 Keynote Live Blog: Skydio and Autonomous Flight
by Dr. Ian Cutress on August 23, 2021 7:00 PM EST- Posted in
- AI
- Trade Shows
- Live Blog
- ML
- Hot Chips 33
- Skydio
- Flight
07:02PM EDT - Welcome to Hot Chips! This is the annual conference all about the latest, greatest, and upcoming big silicon that gets us all excited. Stay tuned during Monday and Tuesday for our regular AnandTech Live Blogs.
07:04PM EDT - Second keynote today is from Skydio
07:05PM EDT - Should be starting here in a second
07:06PM EDT - This talk is about autonomous flight
07:07PM EDT - 'Enabling next generation'
07:08PM EDT - I always think 'autonomous flight' is about flying people, but this is more about the next gen of drones
07:09PM EDT - Starting with the history of drones
07:09PM EDT - First ones were gas powered
07:09PM EDT - Then battery powered
07:10PM EDT - MEMs IMUs and GPUs
07:11PM EDT - Replacing legacy methods cheaper and safer and faster
07:11PM EDT - Cinematography, safety, inspection
07:11PM EDT - 80% of an average drone budget can be consumed by pilot training
07:12PM EDT - confined flying control with ML
07:13PM EDT - looks kinda cool
07:13PM EDT - Moving from age of manual hardware to AI-driven drones
07:14PM EDT - Liken the tradition similar to flip phone to smart phone
07:14PM EDT - now a video showing usecases
07:15PM EDT - First US drone unicorn - $340M funding
07:16PM EDT - World leader in autonomous flight technology
07:16PM EDT - $5.1B market in sales today
07:16PM EDT - 2.3m units
07:17PM EDT - When drones are the size of hummingbirds, what is the market reach?
07:18PM EDT - 100x enterprise growth over 10 years
07:18PM EDT - Today in stage 1/stage 2
07:19PM EDT - Always someone hand-holding the drone right now
07:20PM EDT - stage 3 is basically Drone as a Service - take photos of your kids sports game with an app
07:21PM EDT - Skydio autonomy engine
07:21PM EDT - Sensors, algorithms
07:21PM EDT - 6x4K 360-degree fisheye cameras
07:22PM EDT - That's more than a tesla?
07:22PM EDT - Understand the geometry of the world around it
07:23PM EDT - Depth mapping
07:23PM EDT - Problem is detecting thin structures and wires
07:24PM EDT - understanding the environment
07:24PM EDT - keeping someone in the shot regardless of terrain
07:25PM EDT - generating a 3D view of the world, 3D mapping live and evolving
07:27PM EDT - Simulated examples of use cases
07:27PM EDT - Building 3D maps in real time as detailed as needed
07:28PM EDT - 3D scan sounds cool
07:29PM EDT - Using NVIDA and Qualcomm
07:29PM EDT - Jetson TX2
07:30PM EDT - Qualcomm for communication I'd imagine, or Qualcomm for ISP
07:30PM EDT - Imaging SoC (QC) and Navigation SoC (NV)
07:31PM EDT - Power budget - 100-200W consumes, 20-40 minute flytime. 1W of compute reduces flight time by 10 seconds
07:31PM EDT - Compute constraint is mostly driven by size/weight of the board
07:32PM EDT - Types of compute
07:34PM EDT - Deep Learning inference used in flight - 8 models
07:35PM EDT - It's a big optimization problem
07:36PM EDT - model with predictive control
07:36PM EDT - Motion planning with 6DOF
07:36PM EDT - How the rotor rates should evolve over time
07:36PM EDT - 500 iterations a second with 20+ objectives
07:36PM EDT - 'smooth cinematic flight'
07:37PM EDT - Matrix solvers as per usual
07:39PM EDT - Custom differential geometry library
07:39PM EDT - void handwritten derivatives
07:39PM EDT - high level symbolic python code
07:39PM EDT - Generated code is branchless
07:40PM EDT - Provided an order of magnitude speed-up on CPU
07:40PM EDT - finding ways to apply to other hardware through standard interfaces
07:41PM EDT - Still very early
07:41PM EDT - Real world feedback - foundational building blocks
07:42PM EDT - needs to handle all situations, e.g. sun glare, in a way that's accessible but optimized
07:43PM EDT - Q&A time
07:45PM EDT - Q: Compare to DJI - talk about market is evolving A: key difference with Skydio is in the software and what the user is asking them to do. Industry has been smoke and mirrors - what really matters is true intelligence, need to perform the job and not just be pie in the sky - same with autonomous cars, easy to demo, hard to work 100% of the time or 99.99999%
07:46PM EDT - Q: Call to action - build better processors: Expand, what's the dream? A: Ton of opportunity. Getting the balance of power and functionality is important. Workloads need to be addressed with cameras, more than most, so need to process them with latency and power is key. Do that into a single ASIC is what we need. 2x the compute, but it's all about the size, and 10x would be nice
07:51PM EDT - Q: Sensor suite A: Grad school used LIDAR, today with sensors - LIDARs are orders of magnitude not applicable for the size of drones. Cameras are getting better, Computer vision is getting a lot better, compute is getting a lot better. Exploit it.
07:53PM EDT - Q: Simulated data to train models? How often to retain? A: Becomes very easy (relatively) when you see failure cases. Can mine and exploit data and work into the training set. That data is key. Balancing of data is important. Simulated data is useful, but need real data and bring it together to get to the level of performance you need
07:54PM EDT - Q: Portable algorithms? A: We do specific things that you tailor the software to do, like deep learning toolkits, tensorRT, you would need to reoptimize
07:58PM EDT - Q: Ideal battery? New Tech on the horizon? A: It's a key perf metric. Battery has been getting better slowly over time. Keep abreast over time. Needs aren't crazy - we care more about safety and capacity. Needs are well aligned with laptops and cars.
07:59PM EDT - Q: Hot swap mid-flight? A: Anything is possible. Challenging to do well relative to benefit. Drones could live in docking stations / replace batteries is a better solutions. If you need constant, just use multiple drones
08:02PM EDT - That's a wrap!
2 Comments
View All Comments
eastcoast_pete - Monday, August 23, 2021 - link
One of the biggest potential customer for autonomous drones is, of course, the military. While a couple of countries have signed on to a declaration to not pursue such tech, all major powers have not. Yes, autonomous drones can also do great things, but the temptation of not having to rely on remote control to execute (in every sense of the word) on a strategic objective is enormous. And, with the sky apparently being the limit on budget when it comes to arms, that's where this tech will end up first. Not a pretty picture!abufrejoval - Friday, August 27, 2021 - link
I could not agree more!Except I'm more afraid of the militants than the military.