🎹🎹 Learning Piano in #AR 🎹🎹

👉PianoVision (on #META #Quest2) accelerates the piano learning via Passthrough #AR & hand tracking

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Sheet Insight to learn sight-read
✅MIDI keyboard connectivity
✅Air piano for no physical pianos
✅Multiplayer Music Instruction
✅PianoVision Music Hall in #VR

More: https://bit.ly/3zYvwGX

🈯SAMURAI: in-the-wild Shape/Material🈯

👉#Google SAMURAI: shape, BRDF, per-image pose & illumination. Relightable #3D assets for #AR/#VR.

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Parametrization for varying distances
✅Camera multiplex optimization
✅Posterior scaling of input images
✅Explicit meshes extraction with BRDF
✅Code/data soon available ->#NeurIPS

More: https://bit.ly/3BKWgf3

🦙LaMAR: Localization/Mapping for #AR🦙

👉A new benchmark for #AR in large and unconstrained scenes

😎Review https://bit.ly/3DjlnWU
😎Paper lamar.ethz.ch/files/LaMAR.pdf
😎Project https://lamar.ethz.ch/
😎Code github.com/microsoft/lamar-benchmark

👑 OnePose++: One-Shot Pose 👑

👉Open-source (insane) keypoint-free pose pipeline for #AR

😎Review https://bit.ly/3kF0BdG
😎Paper arxiv.org/pdf/2301.07673.pdf
😎Project zju3dv.github.io/onepose_plus_plus
😎Code github.com/zju3dv/OnePose_Plus_Plus
😎Dataset zjueducn-my.sharepoint.com/personal/12121064_zju_edu_cn/_layouts/15/onedrive.aspx

🧱 MobileBrick: #3D object on mobile 🧱

👉#Apple (+Oxford) exploiting #LEGO bricks to open the most precise #3D dataset ever. Suitable for mobile #AR

😎Review https://bit.ly/3ZqbiAh
😎Paper arxiv.org/pdf/2303.01932.pdf
😎Project code.active.vision/MobileBrick/
😎Code github.com/ActiveVisionLab/MobileBrick

🧩 X-Avatar: Expressive Human Avatars 🧩

👉ETHZ (+Microsoft) unveils X-Avatar: digital expressive humans for life-like experiences (#AR / #VR)

😎Review https://bit.ly/3J6YDLH
😎Paper arxiv.org/pdf/2303.04805.pdf
😎Project https://skype-line.github.io/projects/X-Avatar/
😎Code https://github.com/Skype-line/X-Avatar

🛸 Virtual Occlusions in #AR 🛸

👉Niantic (#pokemongo) on a novel approach for virtual assets to appear ‘sitting among’ the real world objects

😎Review https://bit.ly/3o04wn6
😎Paper arxiv.org/pdf/2305.07014.pdf
😎Project nianticlabs.github.io/implicit-depth
😎Code github.com/nianticlabs/implicit-depth

🥻SF: Towards Virtual Cloth🥻

👉SEA AI Lab unveils a novel #AI to recovery the garment sewing patterns from daily photos for #AR / #VR worlds

😎Review https://t.ly/MwpAV
😎Project https://sewformer.github.io/
😎Paper https://arxiv.org/pdf/2311.04218.pdf
😎Code https://github.com/sail-sg/sewformer

📫MeshPose: DensePose+HMR📫

👉MeshPose: novel approach to jointly tackle DensePose and Human Mesh Reconstruction in a while. A natural fit for #AR applications requiring real-time mobile inference.

👉Review https://t.ly/a-5uN
👉Paper arxiv.org/pdf/2406.10180
👉Project https://meshpose.github.io/