The starting pistol on Apple’s AR exploits was fired when ARKit was announced back in June 2017, and that was only the beginning. Apple is also said to be working on a pair of smartglasses for unveiling in late 2019, with plans to release them to the mass public in early 2020.

So what do we know right now about Apple’s plans? Well, let’s start with hardware. Nikkei reports that Catcher Technology, which makes metal framing and cases for the iPhone, is going to start making lightweight framing for augmented reality devices. It’s not saying it’s doing this for Apple, but the fact that it’s a key partner for Apple makes it an easy leap.

Read next: Best ARKit apps and games

By the way, “lightweight” framing indicates a form factor more like actual glasses, rather than a headset or visor. This would fit in with Apple’s obsessiveness with building the most svelte gadgets possible. It would also match with Apple’s newfound desire to make wearables that are also fashionable, like the Apple Watch.

Apple is looking to power this new device with a custom system-on-a-chip, much like it does with the Apple Watch, according to Bloomberg. These chips, according to CNet, would be able to power an 8K display for each eye. It would also be wireless tethered to a box that houses these chips, kind of like the Magic Leap One.

There’s also going to be a display on it. That display, according to Digitimes, is going to utilize MicroLED. It’s a display technology that uses half the power, is brighter and offers a higher contrast ratio. Plus, it’s thinner, which makes it easier to fit into that lighter frame. Apple is working on a secret facility building MicroLED displays, and though it aimed to debut them in the Apple Watch Series 4, it didn’t. Still, it looks like the plan is to eventually use the tech to its AR glasses.

The display will likely use glass from Corning, a big Apple supplier. Corning patented something called a Wide Field display, which is used for AR and enables a field of view of between 40 and 70 degrees. Seeing as how most AR glasses sport a 45-degree field of view, that’s pretty good.

One interesting note is that it appears Apple may be working on two different AR glasses. In the long-term, it could be working on a pair that works independently, perhaps powered by a box that isn’t a phone or computer, as Bloomberg reported. However, reliable Apple analyst Ming-Chi Kuo says Apple is readying an Apple Watch-like pair of smartglasses for early 2020. These glasses would be an accessory that lean on your phone for power. Seeing how Apple’s chips on the iPhone are extraordinarily powerful, and how successful Apple Watch is, that makes a good deal of sense.

And then there’s the software. It’s reportedly going to run a new fork of iOS called rOS, which stands for Reality Operating System. Apple is currently prototyping AR applications for this new headset, so goes the Bloomberg report. These apps are both rebuilt versions of Apple apps, like Maps and Messages, and brand-new apps like virtual meeting rooms and 360-degree video playback. We don’t yet know how the interface will look or what kind of things rOS will be capable of, though.

We are told that rOS will have an App Store where you can download applications built for ARKit. And interestingly, Apple is internally using an HTC Vive and developing a Samsung Gear-like device to test AR applications for the headset.

This matches up with an earlier Bloomberg report, which says Apple had been in talks with potential suppliers for components of a glasses-like device late in 2016, and “has ordered small quantities of near-eye displays from one supplier” for testing purposes. The device would connect to the iPhone and present images over the wearer’s vision. It would show images and other information in the wearer’s field of vision, and may use augmented reality, the anonymous sources said.

Who’s behind it?

That would be the same team behind ARKit, led by former Dolby hardware head Mike Rockwell, who has assembled a team consisting of veterans from Oculus, Microsoft’s HoloLens team, Amazon’s Lumberyard VR platform, Google Earth, and, perhaps unexpectedly, Hollywood special effects studio Weta Digital, which was behind movies like The Lord Of The Rings trilogy and Avatar.

What's the latest with Apple's AR exploits?

Apple has also moved over people from its camera team to work on the project, which makes sense because the camera is vital toward AR. As AR-focused VC Amitt Mahajan told us, AR not only needs to be good at displaying virtual objects on top of real ones, it needs to be able to see and understand the world around you to do it properly.

So, why 2020? Early in 2017, Robert Scoble said an anonymous Carl Zeiss employee told him the company was working with Apple on a light pair of augmented reality smartglasses that may be announced this year. It turns out that miniaturising the technology to make AR smartglasses that people actually want to wear is incredibly difficult. Don’t take our word for it.

Then CEO Tim Cook had an interview withThe Independent. “There are rumours and stuff about companies working on those – we obviously don’t talk about what we’re working on,” Cook said. “But today I can tell you the technology itself doesn’t exist to do that in a quality way. The display technology required, as well as putting enough stuff around your face – there’s huge challenges with that. The field of view, the quality of the display itself, it’s not there yet.”

Cook continued: “We don’t give a rat’s about being first, we want to be the best, and give people a great experience. But now anything you would see on the market any time soon would not be something any of us would be satisfied with. Nor do I think the vast majority of people would be satisfied.” He did conclude, however, that “Most technology challenges can be solved” but said it’s “a matter of how long.”

Must-read: Meet Apple’s augmented reality dream team

And it would appear that Apple is looking outwards in order to help solve any technological issues. Back in November 2017, Apple reportedly snapped up AR startup Vrvana, who previously developed a mixed reality headset called the Totem. And while it was given the “mixed” label, it was really more of an AR device.

Though it went unreleased and was instead reserved for demos, the Totem would use a couple of pass-through cameras on the outside to view the world and display it on the OLED displays on the inside of the headset. Then it would be able to take the real world and overlay things on top of it, even seamlessly switching to virtual reality if needed.

We’re not suggesting Apple is interested in the VR element of the Totem, but the headset’s ability to potentially shift between use for entertainment and business would fit in with Cupertino’s form. There’s also Vrvana’s work with infrared sensors to consider, as this is something Apple has recently explored through both the iPhone X and AirPods.

Of course, the full responsibilities of Vrvana’s staff are unknown. But something we know will be happening is Apple allowing developers to use ARKit to lay the groundwork for an ecosystem of content, ready and waiting for those smartglasses. After all, we’ve seen some amazing stuff coming from developers recently.

It also purchased augmented reality company Metaio, which was reportedly so sold on Apple’s pitch for AR that it threw the bankers out of the room and wanted to sell itself to Apple for cheap. Apple has been picking up patents that Metaio had been working on more recently.

Apple is also beefing up ARKit’s abilities in iOS so that developers can do more things, like allow apps to remember where you placed virtual objects – so that if you close and open them everything is exactly where you left it.

Apple’s challenges

In November 2017 we reported on word that Apple could launch a mixed reality TV within the next year. It would use the TrueDepth camera system on the iPhone X to be able perform Kinect-like sensing in your living room. There’s also a hint at a larger AR ecosystem built for your home.

However, despite all the apparent fun and games going on, Apple is still trying to figure out a compelling use case for AR moving forward, according to Financial Times and Wired. As it does that, there are multiple prototypes floating around, one of which is as simple as Snap Spectacles, recording video and relying on the iPhone as a display. One thing is for sure: Apple’s AR glasses are 100% real, according to AR-focused VC Matthew Miesnieks, who says he’s spoken to people that have held them.

While Apple is still looking for a use case, a leak on Reddit from alleged Foxconn insiders reveals reveals there could actually be trouble behind the scenes. Chiefly, the employees noted that there’s currently a 65% chance ‘Project Mirrorshades’ is completely scrapped. At the time, the leak said the smartglasses would be delayed until 2018 or potentially 2019.

However, that timeline was even more aggressive than the reported “aggressive” timeline of revealing Apple’s AR glasses in 2019 and releasing them in 2020. And even then, that seems ambitious as we actually think it could be even longer than that before the tech is ready, as per Cook’s comments, delays or not.

With all that said, AppleInsider has reported that Apple was among many industry giants meeting with AR suppliers at CES 2019, which perhaps means that a 2020 release isn’t completely out of the realms of possibility.

According to Bloomberg, in late 2017 Apple was reportedly unsure about how users will actually control the device. It looked at Siri, touch panels and head gestures to get around the device, but hadn’t settled on any of them yet. It even patented a finger-mounted device that would let you interact with objects, kind of like North’s ring for Focals.

The Cupertino company doesn’t typically “lock” device designs in until a year before they ship, so if it’s really going to ship in early 2020 we hope it’s figured all of this out by now.

While it’s clear Apple is still working things out, it’s worth noting that the Reddit leak has been partly corroborated by the Bloomberg report. Thus, it’s worth taking a look at how the report said the glasses could supposedly feature a microphone, accelerometer, magnetometer and a capacitive strip on the arm for volume and call functions. Bone conduction tech would be used for audio, while integration with apps could see head movements control the action. Another interaction method could be Siri, according to some updated patents that included references to the voice assistant as a remote for smartglasses.

Resolution for the glasses is allegedly 428 x 240, with crystal (described as clear), champagne and black designs all being entertained in both men and women’s sizes. Rough estimates also placed the price tag for a potential set at around $600.

Apple everything

  • Meet Apple's AR dream team
    Meet Apple’s AR dream team
    The engineers, execs and researchers building Apple’s secret AR tech
  • The best ARKit apps and games you need to try
    The best ARKit apps and games you need to try
    Got iOS 11? You need to give these a try
  • Apple Watch Series 3 review
    Apple Watch Series 3 review
    An LTE smartwatch that doesn’t phone it in
  • watchOS 4 review
    watchOS 4 review
    Apple’s smartwatch gets more intuitive with the new update

Now, it’s no secret that Tim Cook is excited for AR, and the launch of ARKit proves that Apple is serious about this space. As does the fact that Apple purchased a number of companies that are well-versed in AR software. Or companies like SensoMotoric Instruments, which specialise in eye tracking. Wouldn’t that be helpful for a pair of AR glasses?

That’s what makes the scrapping forecast from Foxconn so interesting. This is a project that has always possessed the potential to be delayed due to the leap forward Apple seemingly wants to make in the space, but a lot is already invested for it to be dropped completely. On the other hand, Apple didn’t hesitate dropping its famous car project. For now, it appears we’ll have to wait for the current moving parts to settle.

If we do see the glasses launched – whatever the case with Foxconn, glasses seem like an eventuality anyway – it would be the third major new hardware category launched under Tim Cook, after the Apple Watch and HomePod. The Apple supremo has also gently nudged away speculation that Apple would be going first to VR, saying that he sees more value in augmented reality. Ol’ Timmy Cook: all about the wearables.

Hot VR headset deals

Oculus Go
PlayStation VR bundle - save $35

PlayStation VR bundle – save $35

Samsung Gear VR (2018)

Samsung Gear VR (2018)

Samsung Odyssey - save $85

Samsung Odyssey – save $85

Wareable may get a commission