How I accidentally built a dev environment that moves with you
24 May 2017 • By Jim Moores
I don't know about you, but I've always been intrigued at how difficult it is to maintain a stream of development, or anything else for that matter, across devices and locations. You work in an office somewhere, go home, or go on-site at a customer's and you find yourself having to manually synchronise everything. Maybe you weren't the world's most diligent dev, and didn't check in after every single edit. Perhaps you've never gotten around to configuring check-style on every single copy of Eclipse you have. Maybe there's a bunch of hard-coded machine-specific settings in your copy of Visual Studio (bad dev!). Maybe you just want to be able to pick up from where you left off! I don't think it's that much to ask. Over the years I've tried many routes to achieving this Nirvana. I think I've finally found one that works. This is the story of my journey to a solution.
For the last year or so, I've been working on a new product, XL4J, which is a library that allows you to write Excel Add-ins in Java. Because it's reliant on Windows technologies, I moved back to Windows as my main development environment after many years on macOS. This was important because I realised that after many years of Apple neglecting developers and making their platform less developer friendly, now macOS really doesn't have anything that Windows doesn't.
- Windows has a bash shell and all the Unix utilities: grep, vim, less, etc.
- You can run proper server software on it (nginx, apache, docker, etc).
- It actually looks quite pretty.
- It has real, full-fat versions of the MS Office suite.
But most importantly, you can cloud-host it. Eh? Surely you must be thinking of the always-almost-but-never-quite-here Microsoft Desktop-as-a-service service? Or maybe you mean some really expensive Citrix set-up involving a minimum hundred grand spend? Or maybe you mean Amazon Workspaces? Hey, that looked interesting. But it was some weird bastardised version of Windows server with some weird PCoIP protocol running over the top (which may be good, I don't know). Apparently, Bezos runs it and it's amazing. I rolled my eyes - one would hope Bezos was able to get a decent virtualized desktop in his own company. When I looked more closely it didn't really fit the developer desktop model.
But it turns out that Microsoft DO have a virtualized desktop service. It's just that it's hidden away in their Azure developer offering and I only found it by mistake.
As part of my development work, I was using Visual Studio and Office so needed an MSDN subscription. For those not in the know, it's how you get useful amounts of MS software for development work - you get developer licenses to all the versions of all the products for a rolling fee. In our case, we applied for, and got, Microsoft's generous BizSpark programme, which gets you a free MSDN license plus some free credits (around $100/month) on Azure for anything up to a few years while you get yourself established.
I had the need to put a couple of servers up, in particular, a Maven repository and thought I might as well use the free credits I had. The first thing that surprised me was that Azure was definitely not Windows only. In fact, it had pre-built VM images for most major distributions and lots of pre-packaged appliance VMs. At first, it seemed pretty complicated and clunky given I just wanted to put up a VM instance but I persevered and got my server up running in a docker container. I didn't really consider using Azure for other VMs as it was more expensive that something like Linode.
One day I was poking around in the portal and I noticed a tab called 'DevTest Labs' had appeared. I started playing around with it and discovered I could create these 'labs' of virtual machines.
It's pretty easy to create one - they have them located wherever you need. I put mine in West Europe, which is actually Ireland. I then discovered how to add a virtual machines to a dev lab:
But the real revelation was the availability of desktop Windows versions to use a VM base image - and without requiring any license keys. This alone would have been really useful. There are also lots of variants in the images, including pre-configured images with Visual Studio, saving you the complexity of manually downloading, installing and licensing them after VM creation. Initially, all this was only for certain versions of Windows, but now they have Windows 7, 8 and 10 making it super-useful for testing.
So after choosing a base image, we need to choose a name for the VM and set up some credentials:
Interestingly, there's the concept of selecting a saved secret rather than putting a password in. Initially, this was command-line only, but it's now in the UI. Last time I tried it I couldn't get it to work correctly, so I used a locally generated password, but obviously, if you're managing a lot of machines it would make sense to look into this more closely.
Now we get to choosing what type of VM to configure:
Now if you're like me, you look at the prices for the VMs and think: wow, that's really expensive. The VM type I'm using most often, DS11_V2 (2 cores, 14GB RAM), is quoted at around £105/month. Don't be too scared by that, it won't cost anything like that if you set it up right. We'll cover that shortly. But first, we also get to choose to install other artifacts on the VM as its image is created. We're used to being able to do this on Linux, but it's much less easy on Windows, until now.
Here I'm adding:
- IntelliJ IDEA (community edition)
- Remote Desktop Connection Manager
- Sysinternals tools
- Chocolatey (and installing filezilla)
Chocolatey is super-interesting here. It's a Windows package manager I'd not heard of allowing an apt-get style install for Windows programs. I get the impression that the package management side of things is a bit of a work in progress and that MS are planning their own package management system. Chocolatey works. Sort-of. Most of the time. I have found that sometimes installation of Chocolatey packages fails during image creation, which can leave a bit of a mess. They do have a paid-for version of the repos that is more reliable, but that seems a bit crap. On the other hand, it still mostly saves a ton of time.
Another great feature, that I won't cover in detail, is that you can create 'Formulas' which are base images + artifacts pre-built. This makes deployment much faster, as the first time you do this it can easily take 10-15 minutes to deploy a VM.
Here, I've sped the process up a fair bit. Just before the creation, note the Advanced settings, which contain some pretty cool details. Firstly, I've used a shared IP address configuration to expose the VM over the internet. If you want to use this seriously, I'd recommend using a proper VPN into Azure, but this lets you experiment freely.
A second interesting feature is an auto-delete option, which allows VMs to expire after a certain amount of time. The most interesting though is the claimable VMs concept. With this, you can put VMs into a pool, and other users on your dev team (or testers, or whatever), can claim and release VMs as necessary. I'm seriously thinking of using it as a nice way to give beta-testers pre-installed VMs to play with.
So once we've kicked off the VM creation, we can look at some of the options. The key things here are Auto-start and Auto-shutdown - these are what help you keep the cost down. Typically I set my VM to auto-start just before I start work and auto-shutdown mid-evening. So they run probably 10 hours a day. But additionally, they only auto-start on weekdays. This means my £100/month comes down to more like £35/month, which I think is reasonable for my main developer workstation. And remember, if I wasn't using it all the time, I could bring it up and down at will, and I often do - it takes 2-3 minutes to bring a machine up from cold. I can also trade up to more cores, more RAM for as long or short a time as I need.
Now we've created a VM, we can connect to it by just downloading the 'Connect' link (which is a tiny .rdp file).
Here I've used the built-in Windows remote desktop client. But I often also use the official Microsoft RDP client for the Mac (called 'Microsoft Remote Desktop') and the iOS and Android equivalents. I will say, I have had trouble using some other RDP clients. Cord wouldn't work and I've had problems with rdesktop too. From what I can tell, this is due to the use of 'Network Acesss Protocol' or NAP, which is an extra layer of security protocol on top of RDP that many RDP clients don't support. It is possible to turn this off but I prefer to just use supporting clients. Oh, another weird thing is that it seems sometimes to prefer not to have 32-bit colour for some reason. Other times it's fine: just switch to 24-bit if you have an issue.
The performance is AMAZING. I mean, seriously, it's like it was local. With no hint of lag (OK, gaming would be a push). You can play video on it as if it was local (RemoteFX helps here), sound just works, you can do remote printing, it supports multiple monitors, full clipboard, desktop resizing. All the little things that make remote access usable in practice.
The closest thing I'd found to this before was LogMeIn, but that was too slow to actually use for development except in an emergency. I've tried VNC too, but it too, was very laggy, perhaps because it's often going via some kind of 'reflector' to avoid a firewall, but I think also because VNC doesn't have the low-level GDI bindings and is pretty much raster-based from what I understand.
But am I like Bezos, sitting in a gigabit paradise accessing a VM located just next door? Um, I wish I was, but I ain't. I'm using this over everything from home broadband to fixed LTE or mobile LTE, and the server is in Ireland (I'm in London). I think the fact it works this well for me means you might be in with a fighting chance of this setup changing how you work too.
To be honest, yes! I can pick up exactly where I left off with development at any time. If I'm working with a customer, I can log into my VM via remote desktop from a laptop. If I'm working at home in front of the TV, I can log in from my laptop there, if I'm working at home I can log in from my home office. And always get the desktop the way I left it (but resized, yay!).
It's even changed the type of laptop I need. I no longer need a quad-core 16/32GB heavyweight - I can just spin up something equivalent in Azure if I need it. All I need is a high-res screen and a keyboard and hopefully now that's all you might need too.
If you enjoyed this article, please take a moment to sign-up to our mailing list for updates and take a look at our XL4J product page. We'd love to hear any feedback you might have.