Every lesson has the key reading built right in (no internet needed for the core concepts. When you want to go deeper or get help with a mini-project, the buttons link back to Claude with context already loaded).
The yellow callouts tell you what changes when you upgrade to Spectacles. The blue callouts have iPhone-specific notes.
Okay, before we do anything cool, we have to make sure everything works. I know, I know — setup is nobody's favorite part. But trust me, getting this right now saves you a lot of headache later. We're going to install Lens Studio, get it talking to your phone, and place our first object in the real world. By the end of this lesson you'll have a chess piece sitting in your room. Yes, really.
What is phone AR, exactly?
AR stands for Augmented Reality. It means taking your phone's live camera view of the real world and adding digital things on top (3D objects, animations, text, particles) that appear to exist in your actual space. When you move your phone, the digital objects stay anchored to where you placed them, like they're physically there.
Think of it this way: your camera is your eyes, and AR puts invisible objects into what your eyes see. The objects aren't real, but they behave like they are.
What is Lens Studio?
Lens Studio is Snap's free desktop app for building AR experiences (called Lenses). It runs on your computer (Windows or Mac). You build your Lens on your computer, then wirelessly push it to your phone where it runs inside Snapchat's camera.
The software stack — what talks to what
- Your Lens: what you build (3D objects, scripts, interactions)
- Lens Studio: the editor on your computer
- Snapchat app: runs your Lens on your phone
- AR framework: technology baked into your phone that understands surfaces, depth, and lighting
On Android and Pixel phones, the AR framework is called ARCore, made by Google. It's deeply integrated into Pixel phones especially. You don't interact with ARCore directly; Lens Studio handles it automatically.
On iPhone, the AR framework is called ARKit, made by Apple. iPhones with LiDAR (iPhone 12 Pro and later) get even better depth sensing. Like ARCore, you don't interact with ARKit directly.
What you will NOT be building
Face filters with silly hats? That's the most visible use of Lenses, but a tiny corner of what's possible. You'll be building world-space experiences: AR that understands and interacts with your real environment.
- What does AR mean? What is it adding, and to what?
- What does Lens Studio do, and where does it run?
- What AR framework does your phone use, and what does it do?
- What is a Lens and where does it run?
- What's the difference between a face filter and a world-space experience?
What you need before you start
- Your computer (Windows or Mac) with a decent GPU
- Your phone with Snapchat installed and logged in
- A Snap account (free, same login as Snapchat)
- Both your computer and phone on the same Wi-Fi network
Step 1 — Install Lens Studio
Go to ar.snap.com/download and download Lens Studio. It's a large download (~2–3 GB). Run the installer and sign in with your Snap account.
Run the downloaded .exe installer. If Windows Defender flags it, click "More info" then "Run anyway" (normal for new software). Works on Windows 10 and 11.
Open the .dmg and drag Lens Studio to Applications. First time opening: right-click → Open to bypass Gatekeeper. Works on Intel and Apple Silicon Macs.
Step 2 — Enable lens preview on your phone
Open Snapchat → tap your profile icon → Settings (gear icon) → scroll to find Lens Studio under "My Account" → enable lens preview.
Open Snapchat → tap your profile icon → Settings (gear icon) → scroll to find Lens Studio under "My Account" → enable lens preview. If you don't see it, update Snapchat from the App Store first.
Step 3 — Your two preview modes
- Emulator (computer preview): a simulated camera view on your screen. Fast for quick checks, but no real depth sensing.
- Live device preview: your actual phone camera in your actual room. Use this for anything that needs real surfaces or depth.
- Lens Studio opens on your computer without errors
- You're signed in to your Snap account in both Lens Studio and Snapchat
- You can see the emulator preview when you open a sample project
- You understand the difference between emulator and live device preview
What you'll build
A lens that places "Hello, World" as floating 3D text anchored to a real spot in your room. When you walk around it, the text stays put. Simple, but this teaches you the full workflow from blank project to phone.
Step 1 — Create a new project
Open Lens Studio. On the home screen, click Snapchat, then click the Default template ("This is a default empty project"). This opens the editor with a blank scene.
Step 2 — Add Device Tracking to your camera
This is the step that makes objects stay anchored in your room. Without it, everything follows your camera like a face filter.
- In the Scene Hierarchy (left panel), click Camera Object
- In the Inspector (right panel), click Add Component
- Choose Device Tracking (not "Device Location Tracking")
- Set the Tracking Mode to Surface
- Check Use Native AR for best tracking quality
You'll see a green grid appear in the viewport, which means Device Tracking is active.
Step 3 — Add a Text3D object
Click the + button in the Scene Hierarchy. Go to 3D and scroll down to find Text3D.
Step 4 — Edit and position the text
- With Text3D selected, find the Text field in the Inspector and type Hello, World
- Adjust color via the Text3D Default Material (white or bright colors read best in AR)
- Set Transform Position: X = 0, Y = 0, Z = -200 (units are centimeters in Surface tracking mode — this places the text 2 meters in front of you)
- If the text looks tiny, increase Scale to 5, 5, 5
Step 5 — Preview and push to your phone
Click Preview Lens in the toolbar to check it in the emulator. Then open Snapchat on your phone, make sure it's on the same Wi-Fi as your computer, and click the push button in Lens Studio. Your text should appear floating in your room.
What success looks like
You see "Hello, World" in your room via your phone camera. When you walk around it, it stays put; it doesn't follow you. That's world-anchored AR.
Troubleshooting
- Text still follows me: Text3D is nested under Camera Object — drag it to root level
- Text is invisible: Try Z = -150 to -300 and Scale 3–8
- Tracking is jumpy: Point your phone at a textured surface — plain white floors are hard for ARCore to track
- Lens doesn't appear on phone: Make sure both devices are on the same Wi-Fi
Part 2 — Your first chess piece
Now that you know the pipeline works, let's swap the text out for something more interesting. We're going to be building a chess game throughout this curriculum, and every great project starts with a single piece. I found this cool low-poly pawn on Poly by Google and we'll be using it as the foundation of everything we build from here.
Go grab it here: Pawn by Poly by Google [CC-BY] via Poly Pizza — click the Download button and save the .glb file somewhere easy to find, like your Desktop.
Step 1 — Import the model into Lens Studio
- In Lens Studio, look at the Asset Browser panel (bottom left)
- Click the + button in the Asset Browser and choose Import, or simply drag and drop the .glb file directly into the Asset Browser panel
- The pawn model will appear in your assets — you should see it listed with a small 3D preview icon
Step 2 — Add it to your scene
- In your Scene Hierarchy, select and delete the Text3D object from Part 1
- Drag your imported pawn model from the Asset Browser up into the Scene Hierarchy; it should appear as a new object at root level (same level as Camera Object)
- If it lands nested under Camera Object, drag it out to root level — same rule as before
Step 3 — Position it in your room
- Select the pawn in the Scene Hierarchy and look at the Inspector
- Set Position: X = 0, Y = 0, Z = -200 (same as the text — 2 meters in front of you)
- The pawn might look enormous or tiny — adjust Scale until it looks like a real chess piece sitting in your space. Try 10, 10, 10 as a starting point and go from there
Step 4 — Push to your phone
Same as before — hit Preview Lens to check the emulator, then push to your phone. Walk around your pawn. It should sit on the floor or wherever you're pointing, anchored in place.
That's your first chess piece in AR. Hold onto this project file — we'll be building on it every phase.
Troubleshooting the import
- Model doesn't appear after import: Make sure you dragged it from the Asset Browser into the Scene Hierarchy — importing alone doesn't add it to the scene
- Model is gigantic: Scale it way down — try 1, 1, 1 or even 0.5, 0.5, 0.5 and work up from there
- Model looks flat or weird: Rotate Y by 90° increments until it looks right
- Textures are missing: Make sure you downloaded the GLB version, not OBJ
Now we're getting somewhere. In Lesson 1 we placed things manually — we basically just said "put it here." Lesson 2 is where AR starts feeling like actual magic. We're going to tap a surface and have the chessboard appear right there, anchored to your table like it belongs. Then we'll write our first real script. Don't panic — I'll walk you through every line.
What's actually happening when AR "sees" a surface
In Lesson 1 you added Device Tracking in Surface mode and things stayed anchored. But what is Surface mode actually doing? Your phone's camera is analyzing thousands of tiny visual details — edges, texture patterns, corners — every single frame. This is called feature point tracking. From those features it builds a rough 3D map of your environment and identifies flat areas it's confident about. Those flat areas are called planes.
A plane is just Lens Studio's word for a detected flat surface — your floor, your desk, a table. Once a plane exists, you can stick objects to it. That's the foundation everything in Lesson 2 builds on.
What a world anchor is
A world anchor is a point in physical space that an AR object attaches to. Once attached, the object stays there even as you walk around — the phone constantly recalculates its position relative to that anchor so nothing drifts. Think of it like hammering an invisible nail into your floor: everything you hang from it stays put.
In Lesson 1 your pawn was anchored at a fixed position you set manually. In Lesson 2 you'll tap a detected surface and anchor the chessboard exactly where you tap, which is much more natural.
What affects detection quality
- Lighting — too dark, the camera can't see features. Natural light or a well-lit room works best.
- Surface texture — a wood grain desk is easy to track. A plain white table is much harder — there's nothing visual to grab onto.
- Camera movement — panning slowly gives ARCore/ARKit time to map the space. Moving fast before it's ready causes tracking errors.
- Reflective surfaces — glass tables, mirrors, and shiny floors confuse depth estimation.
ARCore does all of this purely through the camera — no special depth hardware needed. Pixel phones have some of the best ARCore integration available, so surface detection tends to be fast and reliable. If you're on a plain white desk, put a piece of paper or a book on it first — that gives ARCore something to track.
ARKit works similarly but is more tightly tuned for Apple hardware. iPhone 12 Pro and later have a LiDAR scanner that actively measures depth rather than guessing from visual patterns — surface detection on those models is nearly instant. Without LiDAR, the same slow-pan advice applies.
How this connects to the chessboard
In the next lesson you'll tap a detected surface and the chessboard will appear right where you tapped — anchored to that exact spot on your real table or floor. Understanding what a plane is and what affects detection quality will help you troubleshoot if the board doesn't appear where you expect it.
- What is feature point tracking and what is the phone looking for?
- What is a plane in AR terms?
- What is a world anchor and what problem does it solve?
- Name two things that make surface detection harder.
- What should you do before trying to place an object on a surface?
What you'll build
A lens where tapping your screen drops the chessboard onto whatever surface your camera is pointing at. It stays there anchored while you walk around it. This is the chess game's home base — everything else we build will live on top of this board.
Before you start
Open the project you saved at the end of Lesson 1.3 — the one with the pawn. You'll be adding to it, not starting fresh. Make sure Device Tracking is still on the Camera Object in Surface mode. If you've lost the project, go back to Lesson 1.3 and set it up again — it only takes a few minutes.
Step 1 — Import the chessboard
Download the chessboard GLB: Chessboard by peter moore [CC-BY] via Poly Pizza.
Drag the GLB into the Asset Browser in Lens Studio, same as you did with the pawn.
Step 2 — Set up a Script component on the Camera
Tap-to-place requires a small amount of scripting. Don't worry — we'll keep it minimal here and go deeper on scripts in Lesson 2.4 (the reading lesson right after this one).
- In the Scene Hierarchy, click Camera Object
- In the Inspector, click Add Component → Script
- Click + (Add Script) and choose TypeScript — name it TapToPlace
Step 3 — Write the tap-to-place script
Double-click the TapToPlace script to open it in the editor. Delete everything that's there and paste in the following:
var touchSystem = global.touchSystem;
touchSystem.touchStart.add(function(eventData) {
var hit = global.scene.raycast(eventData.getPosition());
if (hit && hit.isValid()) {
script.board.getTransform().setWorldPosition(hit.position);
script.board.enabled = true;
}
});
Save the script. Back in the Inspector on the Camera Object, you'll now see a Board field under the script component. Drag your chessboard object from the Scene Hierarchy into that field.
Step 4 — Add the chessboard to the scene
- Drag the chessboard from the Asset Browser into the Scene Hierarchy — make sure it's at root level, same as the pawn and Camera Object
- In the Inspector with the board selected, uncheck the checkbox next to its name at the top — this hides it until you tap to place it
- Adjust Scale until the board looks like it's sitting naturally on your table — try 10, 10, 10 as a starting point
Step 5 — Position the pawn on the board
Now that the board will be placed by tapping, the pawn needs to move with it. Make the pawn a child of the board by dragging the pawn object onto the chessboard object in the Scene Hierarchy — the pawn should appear indented underneath it. This way when the board moves to where you tapped, the pawn comes with it.
Step 6 — Push to your phone and test
Push the lens to your phone. Point your camera at a table or floor, give it 2–3 seconds to detect the surface, then tap. The chessboard should appear right where you tapped with the pawn sitting on it. Tap a different spot and the board moves there.
What success looks like
You tap your coffee table and the chessboard appears on it, sitting flat, with the pawn on top. When you walk around, both stay anchored to the table. That's your game board placed in the real world.
Troubleshooting
- Nothing appears when I tap: The surface might not be detected yet — pan slowly for a few seconds first, then tap
- Board appears but flies to a weird position: Make sure the Board field in the script is connected to the right object
- Pawn doesn't move with the board: Check that the pawn is nested under the board in the hierarchy, not at root level
- Board is way too big or small: Adjust Scale on the board object — the pawn's scale may also need adjusting relative to the board
- Board appears tilted: It's inheriting the surface angle — if your table is perfectly flat this shouldn't happen, but slight surface tilt is normal
You already used this in 2.2
The tap-to-place script from Lesson 2.2 used touch input and hit testing without explaining what they were. Now that you've seen them work, let's understand what actually happened.
Touch events
When someone taps the screen, Lens Studio fires a touch event. Your script can listen for these events and respond to them — move an object, change a color, play a sound, start an animation. There are three types you'll use:
- touchStart, fires the moment a finger touches the screen. This is what the tap-to-place script used.
- touchMove, fires continuously while a finger is dragging across the screen. You'll use this for drag-and-drop in Lesson 5.
- touchEnd, fires when the finger lifts off. Useful for "release to drop" interactions.
Hit testing — from screen tap to 3D world
Here's the subtle part: your screen is 2D but your AR world is 3D. When someone taps a spot on the screen, how does the script know what they tapped on in the 3D world?
The answer is hit testing. Lens Studio shoots an invisible ray from the camera through the tap point out into the 3D world — like a laser pointer through the screen. Whatever that ray hits first is what the user tapped. In Lesson 2.2, the ray hit a detected surface plane and that's where the board appeared.
In Lesson 2.5, the ray will hit the pawn itself — and that's how you'll know which piece was tapped to change its color.
The two kinds of hit testing you'll use
- Raycast against world surfaces, which is what you did in 2.2. The ray hits a detected plane in the real world. Used for placing objects.
- Raycast against scene objects; the ray hits a specific 3D object in your scene. Used for selecting and interacting with objects like the pawn.
- What is a touch event and when does it fire?
- Name the three touch event types and what each one is used for.
- What is hit testing and why do you need it?
- What's the difference between raycasting against a world surface vs. a scene object?
Why TypeScript?
Lens Studio uses TypeScript as its scripting language. TypeScript is essentially JavaScript with an extra layer that catches typos and mistakes before your code runs. If you've never coded before, that's completely fine; everything will be explained as we go. You don't need to be a programmer to build great lenses, but scripting is what takes you from "I placed a thing" to "I made a thing do something."
Five concepts you need to know
- boolean: a value that can only ever be one of two things, true or false. No in between; think of it like a light switch (on or off, yes or no). true means yes, it is. false means no, it isn't. We use booleans to track states: is the pawn white? Is it the player's turn? Is the game over? In our next script, isWhite is a boolean that starts as true and flips to false every time you tap.
- variable: a named box that holds a value.
var isWhite = truecreates a variable called isWhite that starts as true. You can read it, change it, and check it anytime in your script. - if/else: makes a decision. "If isWhite is true, do this. Otherwise (else), do that." It's how your script chooses between two different outcomes based on a condition.
- function: a named block of code that runs when called. Touch events call a function when triggered. Instead of writing the same instructions over and over, you write them once inside a function and call it whenever you need it.
- component: an object's properties and behaviors in Lens Studio. Every scene object has components attached to it. The material (color, texture) is a component. The transform (position, rotation, scale) is a component. In the next mini-project, you'll access the material component to change its color property.
Want to go deeper on variables and if/else before the mini-project? These two videos are a great plain-English intro:
How a script is structured
A script is a text file attached to a scene object. It tells that object what to do. Every Lens Studio script has two key moments:
- onAwake runs once when the lens loads. Use this to set things up: initialize variables, connect to components, set starting values.
- onUpdate runs every single frame, roughly 60 times per second. Use this for anything that needs to change continuously over time, like a spinning object or a timer counting down.
How scripts talk to scene objects
Every object in your scene is a SceneObject. Your script gets a reference to any object and then reads or changes its properties. The core loop is: get a reference to something, then do something with it.
In plain English: "Get the pawn object, find its material component, change the color property to black." That's it. Three steps, every time.
You'll also see @input at the top of scripts. This is how you connect a script to a specific object or component from the Inspector panel in Lens Studio, without hardcoding the name in the script itself. You'll use it in 2.5.
- What is a boolean and what are the only two values it can have?
- What is a variable?
- What does if/else do?
- What is onAwake and when does it run?
- What is onUpdate and when does it run?
- What does @input do?
What you'll build
Tap the pawn and it switches between white and black — toggling which player it belongs to. This is genuinely useful for the chess game and also your introduction to TypeScript: variables, if/else logic, and changing object properties at runtime.
A quick TypeScript primer
Before we write the script, here are the four concepts you'll use:
- variable, a named box that holds a value.
var isWhite = truecreates a variable called isWhite that starts as true. - if/else, makes a decision. "If isWhite is true, do this. Otherwise, do that."
- function, a named block of code that runs when called. Touch events call a function when triggered.
- component, meaning an object's properties in Lens Studio. The material is a component. You'll change its color property.
Step 1 — Add a Script to the pawn
- In the Scene Hierarchy, click the pawn object
- In the Inspector, click Add Component → Script
- Click + (Add Script), choose TypeScript, name it PawnColor
Step 2 — Write the color toggle script
Double-click PawnColor to open it. Delete everything and paste this in:
var isWhite = true;
var whiteColor = new vec4(1, 1, 1, 1);
var blackColor = new vec4(0.1, 0.1, 0.1, 1);
var touchEvent = script.createEvent("TapEvent");
touchEvent.bind(function() {
isWhite = !isWhite;
if (isWhite) {
script.pawnMesh.mainMaterial.mainPass.baseColor = whiteColor;
} else {
script.pawnMesh.mainMaterial.mainPass.baseColor = blackColor;
}
});
Step 3 — Connect the script to the pawn's material
Save the script. Back in the Inspector on the pawn, you'll see a Pawn Mesh field. Click the target icon next to it and select the pawn's render mesh component — it should appear in the dropdown as the pawn's visual component.
Step 4 — Test it
Push to your phone. Tap the pawn — it should switch from its original color to black. Tap again — back to white. Tap again — black. That toggle is your first working TypeScript script.
What each line does
- var isWhite = true (tracks the current state, starts as white).
- vec4(1, 1, 1, 1), a color in RGBA format. 1,1,1 is white. 0,0,0 is black. The last 1 is full opacity.
- isWhite = !isWhite, which flips the boolean. If it was true, it becomes false. If false, true.
- mainPass.baseColor, the actual color property on the material. This is what changes visually.
new vec4(0.8, 0.2, 0.2, 1) for red or new vec4(0.2, 0.4, 0.8, 1) for blue. The three numbers are Red, Green, Blue — each between 0 and 1.Lemon Drop 🍋
You've got a working color toggle — now push it further:
- Change the colors to something other than white and black — team colors, your favorite colors, whatever
- What happens if you add a third color and cycle through three options instead of two?
- Can you figure out how to make the pawn start as black instead of white? (Hint: change one word in the script)
- Try changing the color values slightly — what does
vec4(0.9, 0.8, 0.5, 1)look like?
Troubleshooting
- Tapping does nothing: Make sure the Pawn Mesh field in the Inspector is connected — if it's empty the script has nothing to change
- Error in the Logger: Check that you copied the script exactly — a missing bracket or semicolon will break it
- Color changes but looks wrong: The pawn's material might have a texture overriding the base color — try setting the texture slot to empty in the material settings
- Can't find the render mesh component: Click the pawn in the hierarchy, look in the Inspector for a component called Render Mesh Visual — that's the one to connect