Draw with AR in Unity

Imagine painting your imagination into reality and then stepping around it, seeing your ideas come to life from every angle. With the power of augmented reality, you can now bring your own unique art to life and explore it in ways you've only dreamed of. In this answer, you will learn how to create an AR drawing application, opening the door to a world where your creativity knows no bounds.

Setting up canvas

Create a Unity project and set up the AR settings. Now create a canvas by right clicking in the heirarchy window > UI > Canvas.

Then create an empty GameObject called "Reticle" in your canvas. The reticle will be used as an aim for you to draw. Download a reticle image and move it to your textures folder. Change the texture type for the reticle to "Sprite(2D and UI)." Save your changes.

XR origin

In your XR origin, add a plane manager and anchor manager. In this application, the plane manager and anchor manager are crucial in enhancing the user experience and facilitating interactions within the augmented environment.

The plane manager identifies suitable surfaces where the user can draw. By detecting planes, the application can ensure that the drawings are placed realistically within the environment, following the contours of the surfaces.

The anchor manager creates anchors where users start drawing or interacting with the AR space. When a user initiates a drawing, an anchor is created at that point to serve as the starting point. As the user continues drawing, these anchors help keep track of the digital content’s position and ensure that it remains consistent as the user moves around.

To learn more about anchors in Unity AR, click here.

Draw manager

To draw in the detected plane, we'll create a new script called the drawManager. This component will manage the drawing functionality within the AR environment.

Draw manager script

This script allows users to draw lines in 3D space using various input methods like touch and mouse.

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.Events;
using UnityEngine.XR.ARFoundation;
[RequireComponent(typeof(ARAnchorManager))]
public class DrawManager : MonoBehaviour
{
    [SerializeField]
    private float distanceFromCamera = 0.3f;
    [SerializeField]
    private Material defaultColorMaterial;
    [SerializeField]
    private int cornerVertices = 5;
    [SerializeField]
    private int endCapVertices = 5;
    [Header("Tolerancw Options")]
    [SerializeField]
    private bool allowSimplification = false;
    [SerializeField]
    private float tolerance = 0.001f;
    [SerializeField]
    private float applySimplifyAfterPoints = 20.0f;
    [SerializeField, Range(0, 1.0f)]
    private float minDistanceBeforeNewPoint = 0.01f;
    [SerializeField]
    private UnityEvent OnDraw;
    [SerializeField]
    private ARAnchorManager anchorManager;
    [SerializeField]
    private Camera arCamera;
    [SerializeField]
    private Color defaultColor = Color.white;
    private Color randomStartColor = Color.white;
    private Color randomEndColor = Color.white;
    [SerializeField]
    private float lineWidth = 0.05f;
    private LineRenderer prevLineRender;
    private LineRenderer currentLineRenderer;
    private List<ARAnchor> anchors = new List<ARAnchor>();
    private List<LineRenderer> lines = new List<LineRenderer>();
    private int positionCount = 0;
    private Vector3 prevPointDistance = Vector3.zero;
    private bool CanDraw { get; set; }
    
    void Update()
    {
        if (Input.touchCount > 0)
            DrawOnTouch();
        if (Input.GetMouseButton(0))
            DrawOnMouse();
        else
        {
            prevLineRender = null;
        }
    }
    public void AllowDraw(bool isAllow)
    {
        CanDraw = isAllow;
    }
    private void SetLineSettings(LineRenderer currentLineRenderer)
    {
        currentLineRenderer.startWidth = lineWidth;
        currentLineRenderer.endWidth = lineWidth;
        currentLineRenderer.numCornerVertices = cornerVertices;
        currentLineRenderer.numCapVertices = endCapVertices;
        if (allowSimplification) currentLineRenderer.Simplify(tolerance);
        currentLineRenderer.startColor = randomStartColor;
        currentLineRenderer.endColor = randomEndColor;
    }
    void DrawOnTouch()
    {
        if (!CanDraw) return;
        Touch touch = Input.GetTouch(0);
        Vector3 touchPosition = arCamera.ScreenToWorldPoint(new Vector3(Input.GetTouch(0).position.x, Input.GetTouch(0).position.y, distanceFromCamera));
        if (touch.phase == TouchPhase.Began)
        {
            OnDraw?.Invoke();
            ARAnchor anchor = anchorManager.AddAnchor(new Pose(touchPosition, Quaternion.identity));
            if (anchor == null)
            {
                Debug.LogError("Error creating reference point");
            }
            else
            {
                anchors.Add(anchor);
                //ARDebugMenu.Instance.LogInfo($"Anchor created & total of {anchor.Count} anchor(s)");
            }
            AddLineRenderer(anchor, touchPosition);
        }
        else
        {
            UpdateLine(touchPosition);
        }
    }
    void DrawOnMouse()
    {
        if (!CanDraw) return;
        Vector3 mousePosition = arCamera.ScreenToWorldPoint(new Vector3(Input.mousePosition.x, Input.mousePosition.y, distanceFromCamera));
        if (Input.GetMouseButton(0))
        {
            OnDraw?.Invoke();
            if (prevLineRender == null)
            {
                AddLineRenderer(null, mousePosition);
            }
            else
            {
                UpdateLine(mousePosition);
            }
        }
    }
    void UpdateLine(Vector3 touchPosition)
    {
        if (prevLineRender == null)
            prevPointDistance = touchPosition;
        if (prevPointDistance != null & Mathf.Abs(Vector3.Distance(prevPointDistance, touchPosition)) >= minDistanceBeforeNewPoint)
        {
            prevPointDistance = touchPosition;
            AddPoint(prevPointDistance);
        }
    }
    void AddPoint(Vector3 position)
    {
        positionCount++;
        currentLineRenderer.positionCount = positionCount;
        currentLineRenderer.SetPosition(positionCount - 1, position);
        if (currentLineRenderer.positionCount % applySimplifyAfterPoints == 0 && allowSimplification)
        {
            currentLineRenderer.Simplify(tolerance);
        }
    }
    void AddLineRenderer(ARAnchor arAnchor, Vector3 touchPosition)
    {
        positionCount = 2;
        GameObject go = new GameObject($"LineRenderer_{lines.Count}");
        go.transform.parent = arAnchor?.transform ?? transform;
        go.transform.position = touchPosition;
        go.tag = "Line";
        LineRenderer goLineRenderer = go.AddComponent<LineRenderer>();
        goLineRenderer.startWidth = lineWidth;
        goLineRenderer.endWidth = lineWidth;
        goLineRenderer.material = defaultColorMaterial;
        goLineRenderer.useWorldSpace = true;
        goLineRenderer.positionCount = positionCount;
        goLineRenderer.numCapVertices = 90;
        goLineRenderer.SetPosition(0, touchPosition);
        goLineRenderer.SetPosition(1, touchPosition);
        SetLineSettings(goLineRenderer);
        currentLineRenderer = goLineRenderer;
        prevLineRender = currentLineRenderer;
        lines.Add(goLineRenderer);
        // ARDebugMenu.Instance.LogInfo($"New line renderer created");
    }
}

Draw manager script

Explanation

Line 1–5: The script imports necessary namespaces and libraries. These namespaces provide access to classes and functions required for scripting in Unity's AR environment.
Line 9: The script defines a MonoBehaviour class named DrawManager.
Line 10–39: A series of serialized fields are defined to customize the drawing behavior, including distance from the camera, material, vertex counts, tolerance options, and event triggers.
Line 41–52: Serialized fields for the ARAnchorManager, Camera, and default colors are defined.
Line 55–69: Private variables for random start and end colors, line width, LineRenderer instances, lists for ARAnchor and LineRenderer instances, position count, and previous point distance are declared.
Line 71: A property CanDraw indicates whether the drawing is allowed.
Line 75–85: The Update method checks for touch or mouse input and calls appropriate drawing methods.
Line 87–90: A method AllowDraw is defined to set the CanDraw property based on a provided boolean value.
Line 92–101: The SetLineSettings method is defined to configure LineRenderer properties, like width, vertices, and colors.
Line 103–132: The DrawOnTouch method handles touch input-based drawing interactions, which includes creating anchors, adding LineRenderers, and updating lines.
Line 154–134: The DrawOnMouse method handles mouse input-based drawing interactions, similar to DrawOnTouch.
Line 156–166: The UpdateLine method updates lines by adding points if the minimum distance between points is met.
Line 168–179: The AddPoint method appends points to the current LineRenderer, and if simplification is enabled, it simplifies the line periodically.
Line 181–207: The AddLineRenderer method creates LineRenderer objects, associates them with anchors, sets positions, and applies line settings.

Once the script is created, add a default color material. This material will be used for the drawing. Add the anchor manager and camera.

Experience manager

Create a new script to add a custom component in XR origin. It will manage the experience in the AR environment. It helps manage the lifecycle and behavior of AR experiences involving plane detection and initialization.

Experience manager script

It uses the ARPlaneManager component to monitor changes in AR planes. When AR planes change, it activates the experience, invoking the OnInitialized UnityEvent and disabling the ARPlaneManager. It also includes a Restart method to reset the experience by invoking the OnRestarded UnityEvent and enabling the ARPlaneManager.

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.Events;
using UnityEngine.XR.ARFoundation;
[RequireComponent(typeof(ARPlaneManager))]
public class experienceManager : MonoBehaviour
{
    [SerializeField]
    private UnityEvent OnInitialized;
    [SerializeField]
    private UnityEvent OnRetarded;
    private ARPlaneManager arPlaneManager;
    private bool Initialized { get; set; }
    void Awake()
    {
        arPlaneManager = GetComponent<ARPlaneManager>();
        arPlaneManager.planesChanged += PlanesChanged;
#if UNITY_EDITOR
        OnInitialized?.Invoke();
        Initialized = true;
        arPlaneManager.enabled = false;
#endif
    }
    void PlanesChanged(ARPlanesChangedEventArgs args)
    {
        if (!Initialized)
        {
            Activate();
        }
    }
    private void Activate()
    {
        OnInitialized?.Invoke();
        Initialized = true;
        arPlaneManager.enabled = false;
    }
    public void Restart()
    {
        OnRetarded?.Invoke();
        Initialized = false;
        arPlaneManager.enabled = true;
    }
}

Experience manager script

Explanation

Line 1–5: The script begins with the necessary import statements.
Line 9: The script defines a MonoBehaviour class named experienceManager, which appears to manage experiences within an AR context.
Line 11–15: Serialized fields are defined to hold UnityEvents, which can trigger actions based on specific events during the AR experience’s lifecycle.
Line 17: The script requires an ARPlaneManager component to be attached to the same GameObject. This component is essential for managing AR planes in the scene.
Line 19: A boolean Initialized flag tracks whether the experience has been initialized.
Line 22–33: The ARPlaneManager component is retrieved in the Awake method. The planesChanged is added to the PlanesChanged method. This event handler monitors changes in the AR planes within the scene.
Line 27–32: The OnInitialized UnityEvent is invoked, and the Initialized flag is set to true. The ARPlaneManager component is disabled to prevent AR plane tracking in the editor.
Line 35–41: The PlanesChanged method is called whenever the AR planes change. If the experience hasn’t been initialized, the Activate method is invoked.
Line 43–48: The Activate method triggers the OnInitialized UnityEvent, sets the Initialized flag to true, and disables the ARPlaneManager. This method is responsible for initializing the AR experience when AR planes change.
Line 50–55: The Restart method resets the experience. It triggers the OnRestarded UnityEvent, sets the Initialized flag to false, and enables the ARPlaneManager. This method can be called to restart or reset the AR experience.

After writing the script, go back to the XR origin. Under your experience manager, you'll see two fields.

Under on initialized field, add two tabs. In one add the XR origin for your project and choose "DrawManager" > "AllowDraw". Then check the box under it. In the second tab add your reticle and choose the function "GameObject" > "SetActive" and leave the box unchecked. Now, on initialization, you will be able to draw in your application.

Similarly, under the on restarted field, add the same two tabs with the same functions. Alternate the checks of both boxes, so as to disable drawing and enabling the reticle preview on restart.

Demonstration

Free Resources

Learn in-demand tech skills in half the time

PRODUCTS

Mock Interview

New

Courses

Skill Paths

Projects

Assessments

Draw with AR in Unity

Setting up canvas

XR origin

Draw manager

Draw manager script

Explanation

Experience manager

Experience manager script

Explanation

Demonstration

Conclusion