Di Wu
is an AI researcher and educator at the crossroads of AI and robotics. He enjoys using machine learning — from probabilistic graphical models to deep networks — for high-level computer vision: recognising human actions and gestures, tracking, and recovering the full 3D pose of objects: the 2D-to-3D, “inverse graphics” problem.
He received his PhD from the University of Sheffield, U.K., and his bachelor’s from Zhejiang University, China, then did postdocs at the Idiap Research Institute — affiliated with EPFL, in a beautiful Swiss Alpine town — and at Shenzhen University, before moving to industry. He is now at XDOF.ai, and hopes to use machine learning to solve real-world problems.
Bio
Where I have worked and studied.
- Now
I’m currently at XDOF.ai.
- 2022 – 2024
I was a research engineer at Covariant, focusing on AI for industrial robot picking.
- 2020 – 2022
I was a computer vision scientist at FITURE, building real-time embedded AI — detection, key-point recognition and 3D human pose — so a screen could be driven by natural, touchless gestures on a 5-TOPS edge device.
- 2019 – 2020
I was a senior research data scientist at Ping An Property & Casualty Insurance, where I led a computer-vision system for automatic car-damage assessment and used ARCore to render cars in 6-DoF, in real time, on a phone. Along the way our team took 2nd of 864 in Kaggle’s Peking University/Baidu autonomous-driving challenge.
- 2016 – 2019
Postdoc at Shenzhen University on perception for autonomous driving: 6-DoF vehicle pose, instance segmentation and visual tracking — and 1st place in the ApolloScape 3D Car Instance challenge at ECCV 2018.
- 2015 – 2016
Postdoc at the Idiap Research Institute (affiliated with EPFL) in Martigny, a beautiful Swiss Alpine town, on perception and activity understanding. I led person identification in broadcast TV for the EU FP7 EUMSSI project, whose system ranked 1st in the MediaEval 2015 person-discovery task.
- 2013
Research intern at Microsoft Research Cambridge with Ben Glocker, using the Kinect to assess movement disorders in patients with multiple sclerosis.
- 2010 – 2014
PhD in Electronic & Electrical Engineering at the University of Sheffield, on deep learning and probabilistic graphical models for human action and gesture recognition — work that led to papers at CVPR, ACM Multimedia and TNNLS.
- 2006 – 2010
BEng in Electrical & Electronic Engineering at Zhejiang University, where I led the build of a small autonomous car — home-brewed hardware and software — that follows a previously unseen curvy track.
Work
Things I have built, and challenges entered along the way.
-
A production pipeline that turns egocentric recordings into validated, metric 3D hand poses for robot learning. Hands are detected and tracked per frame, initialised with a MANO hand model, then refined by a batched GPU fit — anchored to the stereo point cloud on depth rigs, or driven by 2D reprojection alone on phone video. Temporal smoothing, per-frame quality gates and tiered segment selection keep only trustworthy windows, which are exported as self-contained clips with camera pose for downstream training.
-
Robust detection, key-point recognition and 3D human pose, running in real time on a 5-TOPS edge device so a screen can be driven by natural gestures — “Mission Impossible”-style, or simply ordering at McDonald’s in the time of Covid.
-
Networks built on Mask R-CNN that detect every car and regress its full 3D rotation and translation, then refine the pose with a differentiable neural mesh renderer. 1st in the ApolloScape 3D Car Instance challenge and 2nd of 864 teams on Kaggle.
-
A computer-vision system for automatic car-damage evaluation, with ARCore rendering the car model in real time with 6 degrees of freedom on a phone.
Honours & challenges
- 2nd 2020
- 1st 2018
- 3rd 2018
- 23rd 2017
Future Challenge — Helping Balloons Navigate the Weather
- 7th 2015
- 1st 2015
- 8th 2012
Papers
16 publications, newest first.
