My First Hologram

20160520_081748_HoloLens

When Microsoft first announced the HoloLens, I was interested in the possibilities that this device opened up. When I tried it a few months later at Build 2015, I knew I had to get my hands on it and start to developing for it.  This past winter, Microsoft announced that they were going to start selling developer devices so that developers could start getting apps into the store. Microsoft wants a vibrant app store before these devices go to consumers. Of course I had to sign up to get one. Since there is a limited number of devices being manufactured, Microsoft set up distribution in waves.  Developers in each wave could order 2 device when their time came.  They also said that it would take 2-3 months for each wave to be served. I was in wave 2, so the wait was on.

Last week I received my notice that it was my turn.  I ordered the device and got it a few days later. That was last Thursday. So I did what everyone does when they get one, I played.  There are some pretty cool apps already available for the device including the community suggested, Microsoft built, Galaxy Explorer. You can even get the source code if you want to see how the magic is done. Once I got the playing out of the way it was time to get down to business. I don’t want to just play with holograms I want to build them.

Lately Microsoft has been building great developer resource websites to help get developers started on their technologies. That is true for HoloLens. I encourage you to pop over to the Holographic Developer Center and check all the resources available. There are two types of apps you can build for the HoloLens, 2D and 3D (holograms). 2D apps are just Windows 10 Universal Apps. So if you can build a Universal Windows App then you can build a HoloLens App. To load the HoloLens tools and emulator you will need Visual Studio 2015 Update 2. If you want to build 3D apps you will need a different tool.  The easiest way is to use Unity3D but not just any version of Unity3D, a special beta version. Note the word special, more on that in a bit.

hololens errorThe SDK, the emulator and the special beta version of Unity were released at this years Build conference in San Francisco. I was downloading the tools before the keynote announcing the was done.  I wanted to start to play. I got things up and running with relative ease. If you don’t have a device the emulator is pretty good. Fast forward to this past weekend when I wanted to start developing for the device. I figured I would use the app I got running in the emulator. No Joy! Try it in the emulator again. No Joy! So I Google’d it with Bing and found that other people were having similar issues but not just with HoloLens apps with Win 10 apps as well. The solution reinstall the tools.

Keep in mind there are a lot of beta bits/special configurations required to develop for HoloLens. We are early on in the process, on the cutting edge so not all of the tooling is locked in.  If you are like me and develop for many different platforms and experiment with many beta bits, your Visual Studio can get out of whack. I did not want to reinstall Visual Studio so I went about tweaking, big mistake. Come to find out I couldn’t even build and deploy UWP apps to my local machine. Step one fix UWP. Step two deploy 2D UWP app to HoloLens. To accomplish this I ended up having to repair my Visual Studio instance. Not as bad as reinstalling but still time consuming. Mission accomplished.

Now onto holograms. I decided I would not try to wing it so I took a step back and visited Microsoft’s Holographic Academy.  While I was out at Build this year I got to do a half day Holographic Academy session and it was a great way to get into developing for the HoloLens. More specifically I visited Holograms 100: Getting Started With Unity. This tutorial walks you through setting up Unity for HoloLens development. At the end you have a cube floating in your environment (the Hello World for holograms). I had the special version of Unity so I was ready to go.  I started the tutorial. I try to deploy to the HoloLens….No Joy! Back to Bing.  It seems that the special version of Unity has been updated since Build.

Ok time to uninstall and reinstall Unity and people on Bing said beta 16 is the flavor of the month. Get it on my machine. Let’s get this cube floating…..no! Now what? I go back and walk through the tutorial again and realize that the virtual SDK is not set correctly.  It should look like this:

UnitySDK

Come to find out it is not beta 16 I need it was beta 16-HTP.  Lesson learned: follow the links on the Dev center page to ensure that you are using the latest bits.

Ok time to uninstall and reinstall Unity……again!! Right! Here we go. Redeploy and…..a 2D app running the Windows 10 unity player. There is my cube but it is not 3D.  Why can’t I get a cube to float in front of me? I am going on 2 days of this. All I want to do is see a floating cube. Is that too much to ask for?  Come to find out I skipped a step in the tutorial:

ManifestSettings

This seems important. Don’t you think? So after 2 days of tinkering and uninstalling and reinstalling Unity and repairing Visual Studio a couple of time….success (see picture above).  A floating square!

Why am I excited about a floating square? I have no Unity experience so I am learning as I go. This exercise has taught me a few things.  First Don’t be afraid to reinstall the tools if they weren’t working. I think that my initial reaction to having to do that kept me going down a path of try this/fail to the point where the time I wasted tinkering could have been spent developing. On the plus side I did learn about all the different settings that I need to troubleshoot in the future. Next, it is early on in the development cycle and the tooling is still being developed. So things that you would expect to be done for you may not be done for you. Case in point the manifest change that you need to make. Finally there is not a lot of help outside of the community page for HoloLens development.  It is that early so start there.

So where do I go from here.  I want to do the other tutorials in the Academy.  They address the capabilities of the SDK.  Next I will spend some time poking around the documentation to get a better feel for what is available. To me the compelling enterprise examples will require getting external data in to my apps. So I will look into/develop some best practices around that. How do I get data into my Unity app? How do I make it interactive? Finally I need to learn how to really use Unity beyond floating cubes. I see Unity training/tutorials/experimentation in my future.

I am excited about where HoloLens is taking computing.  I can only imagine where we go from here.  In the mean time I get to build holograms and I think that is pretty cool.

MagicKingdomEntrancePicture

Anyone who has visited the Magic Kingdom in Disney World, will recognize this sign that hangs above the entrance.  I believe that this can applies to the world that HoloLens opens.

Posted in HoloLens | Leave a comment

The World of Holograms Approaches!

hero-hololens_845x341

Yesterday Microsoft announced the availability of the HoloLens Developer Kits.  The kits are available to developers who registered and were approved to get them. They will be shipped in waves with wave 1 shipping on March 30th.  These kits are going for $3000 and are only available in the US and Canada. Making a real hard decision on whether or not you should buy or not.

Microsoft HoloLens is the world’s first fully untethered holographic computer. HoloLens redefines personal computing through holographic experiences to empower you in new ways. HoloLens blends cutting-edge optics and sensors to deliver 3D holograms pinned to the real world around you. Microsoft Hardware site

Microsoft’s goal with this release is to get devices into the hands of developers in order to build up a supply of applications in the store. To that end they made the developer site available yesterday as well.  Keep in mind that this is still early on in the process so there are a lot of placeholders throughout the documentation, including the additional SDKs and HoloLens Emulator. It looks like more information will become available March 30th, which, coincidentally is the first day of Build 2016. Since all of the content is usually made available online and some even streamed live, like the keynotes, I would tune in to see what happens. 

The Hardware

Amongst the details released yesterday were the hardware specifications.  There are some pretty cool breakdown photos on the Hardware site. Here is what we know taken from the hardware site:

Optics

  • See-through holographic lenses (waveguides)
  • 2 HD 16:9 light engines
  • Automatic pupillary distance calibration
  • Holographic Resolution: 2.3M total light points
  • Holographic Density: >2.5k radiants (light points per radian)

Sensors

  • 1 IMU
  • 4 environment understanding cameras
  • 1 depth camera
  • 1 2MP photo / HD video camera
  • Mixed reality capture
  • 4 microphones
  • 1 ambient light sensor

Human Understanding

  • Spatial sound
  • Gaze tracking
  • Gesture input
  • Voice support

Input / Output / Connectivity

  • Built-in speakers
  • Audio 3.5mm jack
  • Volume up/down
  • Brightness up/down
  • Power button
  • Battery status LEDs
  • Wi-Fi 802.11ac
  • Micro USB 2.0
  • Bluetooth 4.1 LE

Power

  • Battery Life
  • 2-3 hours of active use
  • Up to 2 weeks of standby time
  • Fully functional when charging
  • Passively cooled (no fans)

Processors

  • Intel 32 bit architecture
  • Custom-built Microsoft Holographic Processing Unit (HPU 1.0)

Weight

  • 1.28lbs (579g)

Memory

  • 64GB Flash
  • 2GB RAM

This is a pretty impressive set of hardware considering it is all contained in headset that weighs less than 1.5lbs. Keep in mind that this is the developer kit.  The final release hardware (believed to be a couple of years away) will be different/better

The Developer Kit

So for $3000 dollars, what do you getMSHoloLens_GroupShot_wAcc_WhtBG_V2_RGB-1024x444

  • HoloLens Development Edition
  • Clicker
  • Carrying case
  • Charger and cable
  • Microfiber cloth
  • Nose pads
  • Overhead strap

The Developer Experience

To develop for HoloLens you are going to need the following:

  • Windows 10 PC (64bit)
  • Visual Studio 2015 Update 1
  • Unity (5.4)

You will build Universal Windows Apps with Visual Studio, using Unity to build actual holograms. You can find all the details, along with guidance,  in the developer documentation. Remember this is early on in the process and not meant for production so there are some limitations in the APIs that are available.  Microsoft has said that they are currently working to close those gaps and will make additional APIs available in future updates. The goal is to have the majority of Windows 10 UWA run on HoloLens. 

My Thoughts

After a quick run through of the available documentation one thing jumped out at me, only one application can be running at a time.  Holographic applications can’t interact with each other. This seems like a limitation that they will work to reduce as most of their promotional videos show more than one app running at the same time. I hope they are able to fix this in the future.

What does all this mean? The way we interact with computers is about to change.  Will this breakthrough become the norm? Only time will tell. Microsoft is building a platform the allows developers to explore the future of computing and is changing how we think about what experiences we want our users to have.

Posted in HoloLens, Windows 10 | Tagged , | Leave a comment

Azure Event Hub REST API

imageI was recently worked on a project where I had to use a mobile device, specifically Window 8.1 phone/tablet, as a hub for an external sensor. The mobile device then needed to pass the data to an Azure Event Hubs endpoint. The data was then going to be sent through Azure Stream Analytics to various storage solutions. Normally you would fire up a project, go to NuGet package manager, install ‘Microsoft Azure  Service Bus’ SDK, set your endpoint URL and send key, and be off and running. There is one slight problem with that in my architecture. One of the SDK dependencies is not compiled against WinRT so you are out of luck there.

Luckily Azure Event Hubs exposes a REST API that is fairly straight forward except for one thing. How do I assure the service I have the appropriate authorization to send messages. The REST endpoint expects a Shared Access Signature in the header of the request. Here is some sample code for creating that token:

image

Basically it is a string containing the base URL a hash of the base URL and expiration (using the send key as the hash key) and the key rule name.  You can go to the MSDN documentation for more information on Shared Access Signatures.

I created a little proxy class that encapsulates the generation of this token and handles sending the message:

image

My Event Hub expects all messages to be formatted as JSON. My proxy method takes the serialized object and packages it up as the content of the HTTP request. It also sets the SharedAccessSignature header value.

UPDATE: I recently got asked what [PATH TO MESSAGE ENDPOINT] is,  So here is a little explaintion. In the EventHubProxy you have [BASE SERVICEBUS URL] which should look something like this: https://xxxxxxxxxx.servicebus.windows.net, where xxxxxxxxxx is the name of your services bus namespace. On the portal website you should see you EventHub Url which looks something like this https://xxxxxxxxxx.servicebus.windows.net/zzzzzzz, where zzzzzzz is the EventHub name.  So  [PATH TO MESSAGE ENDPOINT] is ‘zzzzzzz/messages’. I hope this clears it up.

Sending and event to the hub is as easy as:

image

One thing to note in this architecture is that mobile devices tend to be on metered connections. So be aware that sending loads of data will count against the devices data plan. You may want to restrict sending data only when the device is not on a metered connection.  You can get the code and play with it.  I put TODO statements where you have to provide you environment specific values.

Posted in Architecture, Azure, Cloud, IoT | Tagged , , | Leave a comment

My Visual Studio 2015 Review

Visual-Studio-2015

Recently I was asked by SDTimes to write a review of Visual Studio 2015.  I had been using it since early previews so I was more than happy to give my two cents.  Check out the article Visual Studio 2015: An insider’s review and tell me what you think.

Posted in Uncategorized | Leave a comment

A Universal Example

In my last post What Does It Mean to Be Universal, I talked about Microsoft’s new Universal Application Platform (name may be changing to Universal Windows Platform), coming in Windows 10. This new application platform allows you to build one application and run it on different platforms.  In that post I told you how Microsoft was doing that. In this post I want to look at an example of what a single app running everywhere, might look like.

Scenario

image

Let’s say you were building an application for manufacturing company. They want to deploy IoT sensors running Windows 10.  These sensors would gather telemetry data from the machines on the factory floor.  This data would be transmitted to the cloud where it would be analyzed and reported through machine learning (out of scope for this post).  The data then can be accessed on Windows PC, Tablets and Phones. They also want to take advantage of augmented reality technology to give their plant manager real time access to the data while they are walking around the plant.

The Old

imageIn the past you would probably create a project that looks something like this. You would have multiple projects for each head you want to support. Plus you would have some libraries for code that you want to share across each of the heads. After compiling, you end up with an application for each head. Although not too complex, there is the potential for major code duplication. This is especially true when it comes to the interface and platform specific code (where API differ). There was less of an opportunity for reuse.

The New

imageUnder the system you can create one project, compile into one application, and have it run on all the devices. The APIs are pretty much the same.  When they do differ for a device, Microsoft has a way for that code to still live in the same project (see my last post for a sample). There may be times when you want to separate out code. For instance code that can be reused in projects that are not part of the new platform, server code comes to mind.  That should still be possible.  I say should because we don’t know for sure. We will have to see what comes out at Build next week. Hopefully we will have more clarity on this.

Thoughts

So one project or many projects?  This universal platform is going to make us rethink how we architect our solutions going forward.  Does it make sense to have everything in one project or do you want to keep things separated? I can definitely see creating utility projects that get reused between different solutions but does it make sense to have a .dll for Services or Models?  You used to be able to replace those components just by swapping out dll’s but now everything gets wrapped up into an apex package that is signed.  Swapping out dll’s would invalidate the package so a new build is needed. I am interested in hearing what people have to say about the new universal platform.

Posted in IoT, Software Architecture, Universal App Platform, Windows 10 | Leave a comment

What Does It Mean To Be Universal

So what does it mean to be universal when it comes to Windows 10?  A new way of building applications is coming in Windows 10. The past few years Microsoft has been on this path of converging all of their operating systems to run on a common core.  Windows 10 is the culmination of this journey. One of the benefits of having a common core is the ability to create an API that will allow apps to run across all the devices that use this core.  So Microsoft introduced the Universal Application Platform (UAP).  Gallo blog 1 v2

 

The UAP will allow developers to build applications that will run across the One Windows Platform. There are many challenges to achieve this goal of write once and run on all these devices. There are all kinds of devices that these apps can run on from phones with 4” screens, to running on Xbox with 52” screens, to devices with no screens at all. These different device have different capabilities. So how do you address these needs.

How Did We Get Here

Back in the day Microsoft’s “cross” platform solution was to create a portable class library. These libraries were restricted to the least common denominator of frameworks you want to address. This created a dll that could be referenced by the reference frameworks. This was great except that you were restricted on which API’s you had access to plus you ended up with multiple applications.

With Windows 8.1, Microsoft introduced a universal application template that helped developers build phone and modern Windows 8.1 apps. A “minor” convergence of the API happened, bringing the phone and the tablet closer together. The converged API allow for more code reuse and minimal device specific code. It did this by creating a shared project that injected the common files into each project (Windows and Phone). So you could share a lot of code but in the end you still have two applications. And the code you write for these universal apps was specific for the phone and modern applications (formally known as Metro).

Moving Forward

In Windows 10, Microsoft is taking the concept of universal apps from Windows 8.1 and expanding on it to cover a wider array of devices. The common core operating system was the first step. At the base of the Universal Application Platform is this thing call the Universal Device Family. This is the set of APIs that are guaranteed to be available to all applications written with UAP.  Once an UAP app is compiled there is one app package that can run on all these different device families.

Device families

From the diagram above you can see that there are many device families that your application can target (one, some, or all). Does that mean there will be Xboxs API installed on my desktop? There has to be if I expect my code to compile, right? Also how do I take advantage of device specific functionality, like the hardware back button on my phone? Hey some of those IoT devices are pretty small to be having all these APIs installed, remember we are creating one package to rule them all.

Microsoft handles this through Extensions. When you need to target specific functionality tied to one of these device families, you add a reference to the extension for that device family. Those extensions are just metadata files with the available API’s for that device family. When the application compiles it does so against these metadata files.  The actual API’s will be installed on those specific device families.

ApiInformation

Now that I have those references I can start writing code for the phone back button and expect that code to run just fine on the Xbox right? Well not quite. You don’t want to try to call an API that is not implemented on you specific device family where your app is running.  You will get all kinds of run time errors. How do you handle these API calls?

In the past when we had shared code, we would isolate device specific code in #IF statements. But wait you said there would be one app package and #IF suggest multiple since it is a compiler directive. In Windows 10, you do not want to query for device types. Device types will be added all the time and your #IF statements could get out of control. Instead, Microsoft, has provided a means to query for capabilities/API.  They have introduced the ApiInformation class. 

The ApiInformation class allows you to test for the presence of the API you want to call. Here is an example checking to see if there are phone hardware buttons present.

image

On thing to notice is the use of Magic Strings.  Those are bad. I hope that Microsoft comes up with a better way to do that. Maybe have a constants file that contains the most common types users are going to query for and the leave it up to the user to handle the exotic cases. Either way there is a good chance that you will fat finger a fully qualified name. It is an early release of the SDK so things can and most likely will change.

By querying for specific capabilities, line 39 will only run if there are hardware buttons present.

The …

The beauty of the UAP approach is that it is extensible.  Did you notice “…” in the diagram above.  They are leaving it open for new platforms in the future, say for instance HoloLens. Or even devices that don’t exist today.  If you install the preview SDK you will see that there are only 2 extensions present, one for the desktop device family and one for the phone device family. Expect more to come in future releases.

Final Thoughts

The direction that Microsoft is going is exciting. It will be interesting to see how they follow through on this One Windows Platform vision. It will also be interesting to see if someone like Xamarin can take advantage of the UAP and extend it to iOS and Android.  Hopefully we will get more insight at Build this year.

Posted in IoT, Modern Application, Software Architecture, Windows 10, WinRT | 1 Comment

Layout Awareness in Windows 10 UAP

There have been lots of changes with Windows of the past few years. A while back, Microsoft radically changed the way Windows worked with Windows 8. The goal was to start to converge the different operating systems into a single core. Windows 10 is the culmination of that convergence which started out as “Three Screens and a Cloud” and now includes many more screens. 

image

The road to convergence is a difference story. Here I want to talk about the user experience. While all this was going on there was a revolution in the experiences that users expected. No longer are battleship gray user interfaces acceptable.  Neither are shrunk down versions of these interfaces acceptable on mobile devices.  As a developer you need to design and build you applications for the form factors that you are targeting. Here is where we pick up that journey on the Microsoft stack.

The Past

In Windows 8 there were two environments, a desktop environment, and a tablet environment and with the tablet environment they introduced a new programing language WinRT.  Those tablet apps, sometimes referred to as “Metro” apps, could be running in one of four states; Filled, Full, Snap and Portrait. There was a neat little enum that helped support these states.  One of the mantras was good design. Your app should look good and respond well/do the right thing in any of those states.  To assist developers, the Visual Studio template for those projects included a LayoutAwarePage that assisted in handling the transition between states.  I wrote a blog about it What is This ‘LayoutAwarePage’ You Speak Of.

Along comes Windows 8.1, that enum goes away and so does your LayoutAwarePage. In 8.1, apps could be resized horizontally independent of those defined states. There was another wrench thrown into the mix, the phone.  With this release WinRT apps can be developed for the phone. The templates changed to include a new Universal App template.  There was guidance from Microsoft that you should target both table and phone but no tool build in the framework to help. At Build 2014 there was a pretty good session on how to target screens of any size: From 4 to 40 inches: Developing Windows Applications across Multiple Form Factors.  During this session Peter Torr, showed the science of view items on different size screens, he also showed a potential solution to the lack of built in tooling. Since I wrote about the LayoutAwarePage in 8 I figured I better write a post about where in went in 8.1: What Happened to My LayoutAwarePage?.

The Now

In Windows 10, Microsoft wants to target a larger array of device, including some that don’t even have screens. From the image bellow there are a plethora of platforms that your apps can run on.  The convergence allows you to write code that runs on all these devices but does that mean I have to write a separate UI for all these devices?

The answer is no. Taking a page from web design, the new guidance is to build adaptive interfaces.  Keeping in mind that the experience should be tailored for each specific form factor.  Recently a preview of the SDK for Windows 1o was released along with a Microsoft Virtual Academy training course. Module 9 caught my attention because it talks about building adaptive UI and it looks very similar to the solution that Peter Torr came up with for 8.1. In Windows 10 you can truly build one app (one app package) that will run on any Windows 10 device. There is some neat magic behind the sense that allows this to happen and if you are interested in that you should watch the other modules in the course. To support writing a single Xaml file that runs across devices, Microsoft has revamped Visual State Manager to assist with building adaptive UI.  I highly recommend that you take a look at that module if you are interested in the story.

I am sure that more details will come during Build 2015 (April 28-May 1) so keep an eye out for what coming down the pike. Keep in mind that everything is in preview right now so things may change between now and release but this looks promising. 

Posted in Mobile, Modern Application, Universal App Platform, Windows 10 | Leave a comment