Free, automated SSL certificates for Sandstorm self-hosters
By Asheesh Laroia - 01 Oct 2015
TL;DR: If you install Sandstorm and choose to use a subdomain of our dynamic DNS service sandcats.io, we’ll automatically set you up with an HTTPS certificate from GlobalSign. The private key never leaves your machine and is automatically rotated every week.
People who run their own servers often do it for privacy. Many of them use our open source Sandstorm software to easily run apps like document editors, group chat apps, task management, and more.
But a private server is not really private if you’re communicating with it in plaintext. So, starting today, if you install Sandstorm on your own Linux server, and you choose to host at a subdomain of our free dynamic DNS service sandcats.io, you will have free, automatically-renewing HTTPS certificates by default. That means you get privacy on the network without any hassle.
For companies and individuals who run their own Sandstorm servers, it’s now easy to secure yourself with HTTPS. Here are the steps.
-
Download and run the Sandstorm install script.
-
Choose a subdomain of
sandcats.ioyou want to register (and tell us how to contact you to recover the domain). -
Visit your server’s new web dashboard by clicking the URL printed out by the install script.
Sandstorm automatically enables HTTPS as part of step 2. There’s no setting up reverse proxies, no creating certificate signing requests, no waiting on a CA. Your private key is on your machine and never leaves.
Why you need HTTPS
HTTPS (also commonly but incorrectly called “SSL”) is important because the web’s original protocol, HTTP, is insecure – anyone in a coffee shop near you can see what you’re browsing or modify your traffic. This is a real threat; here’s a video of someone stealing HTTP traffic that was made as a school project.
Over the past five years, web companies like Google, Facebook, and Twitter have been switching their sites to require HTTPS. Sandstorm.io and all of its subdomains have always required HTTPS. The Chrome team is planning to show a broken padlock for plain HTTP sites. Companies that are responsible for a lot of user data have been making time to secure their sites with HTTPS.
Our search for a certificate vendor
Providing automatic HTTPS is more than just a technical problem. To be trusted, the certificates must come from a Certificate Authority that is already trusted by your browser. To deliver certificates to you, we needed to work with such an authority.
So, earlier this year, I began to talk to people at CAs. But there was a complication: Sandstorm requires a wildcard certificate to achieve some of its security properties. That is, if your server is example.sandcats.io, all subdomains *.example.sandcats.io must be covered as well. Wildcard certificates are usually expensive – an order of magnitude more so than normal certificates. And when I talked to the inspiring folks at Let’s Encrypt, I discovered that they weren’t ready to issue wildcard certificates.
I was lucky enough to find Michael Trotta at GlobalSign. He believed in our mission of making servers safe and easy for more people to run. Working together, we were able to set up a deal allowing us to offer free wildcard certificates under sandcats.io. Contracts were signed, and then it was time to write some code.
Defending against the next Heartbleed: Weekly key rotation
We want Sandstorm to require as little maintenance as possible, so that there’s no reason not to run your own server. We especially want security problems to be solved with no action on your part.
One of the biggest headaches for server admins in recent memory was Heartbleed, a bug in the popular OpenSSL library which could have revealed people’s private encryption keys to attackers.
Patching the bug is the easy part: If you have automatic updates enabled, your Sandstorm will be updated within 24 hours of any release without any action on your part.
What made Heartbleed so bad, though, was not just that systems had to be updated, but that all HTTPS keys and certificates had to be revoked and regenerated, in case they had already been stolen. This was a huge pain for sysadmins, and a huge pain for the certificate revocation infrastructure, which was not designed for everyone in the world to revoke certificates simultaneously.
To solve this, Sandcats issues certificates valid for only seven days. Your server will automatically generate a new key and receive a new certificate every week. So, any compromise of your private key “fixes itself” over time. Granted, a week is a long time to live with the possibility of an active MITM attack, but realistically it took much longer than that for most revocations to take effect after Heartbleed (if they ever took effect at all).
The result: Every Sandstorm app is secure for self-hosters
Because Sandstorm handles HTTPS, rather than the app or the system administrator, the Etherpad package in Sandstorm doesn’t need to have any kind of special integration, nor does the Rocket.Chat package, nor does the Brainstorm package, nor any other.
So take a moment and install Sandstorm now to securely enjoy these kinds of apps, or at least try the online demo.
And if you already have a Sandcats-enabled install of Sandstorm from before today, check out the documentation to enable HTTPS!
Is curl|bash insecure?
By Kenton Varda - 24 Sep 2015
Since the early days of Sandstorm, we have offered the following mechanism for installing it:
curl https://install.sandstorm.io | bash
As it turns out, a lot of people object to curl|bash. In fact, some people object very strongly to it, going so far as to scoff publicly at how obviously terrible it is and smugly proclaim that they dismissed our entire project when they saw this line. Usually, the loudest objectors claim that curl|bash is bad for security.
Sandstorm is a security product, so we want to address that head-on.
The Bogus Argument: Code Execution
I’ll admit: curl|bash certainly smells funny. To anyone with a basic understanding of Unix, the construction makes it really obvious: This command will give the named web site direct access to your system, with the ability to do anything that you could do. This feels very wrong: We should be able to install software without giving the developers full access to our systems, right?
We at Sandstorm would obviously be the first to agree that software you install should not be automatically fully trusted – that’s why Sandstorm itself runs every app in a secure, isolated sandbox by default. Unfortunately, however, traditional Unix software is always granted the full authority of the user who runs it. When you install software on Linux, no matter what package manager you use, you are giving that software permission to act as you. Most package managers will even execute scripts from the package at install time – as root. So in reality, although curl|bash looks scary, it’s really just laying bare the reality that applies to every popular package manager out there: anything you install can pwn you.
If you wish to install Sandstorm – or any software – without giving it full access to your system, you must install it on a dedicated machine, VM, or (perhaps, with caveats) user account. In fact, we highly encourage you to do so, for defense in depth. But, we know it’s more work than a lot of people want to deal with.
The Better Argument: Code Signing
Some of the objectors, though, go a bit further: They claim that curl|bash is more open to attack that other distribution mechanisms, potentially allowing a third party to replace our Sandstorm downloads with evil ones.
Of course, all content served by sandstorm.io – from software downloads to our blog – is served strictly over HTTPS (with HSTS).
Various people argue, though, that this is not good enough, and that we should be signing each release with keys kept offline.
In fact, I agree: where possible, we like to use two or more independent mechanisms of protecting against a security problem, so that when one breaks it’s not a big deal. So, as of today, it is possible to verify the Sandstorm installer from the PGP keys of various Sandstorm developers by following our “PGP-verified install” instructions.
That said, we do not think this issue quite warrants the visceral anger we’ve seen people express over curl|bash. Realistically, downloading and installing software while relying on HTTPS for integrity is a widely-used practice. The web sites for Firefox, Rust, Google Chrome, and many others offer an HTTPS download as the primary installation mechanism. It’s even the standard way to install most Linux distros in the first place (by downloading an iso from the web site). Many popular package managers, such as npm, rely only on HTTPS for integrity. curl|bash over HTTPS is just as “secure” as any of these.
Verifying Keys with Keybase
To get benefit from PGP code signing beyond what you get from HTTPS distribution, you must verify that the software is signed by the specific developer you are expecting – and to do that, you need a way to bootstrap trust in their PGP key. This means you can’t simply fetch the key from the developer’s own server. You can use PGP “Web of Trust”, but sadly few people really understand how to do that.
Recently, though, a fairly plausible option has emerged: Keybase.io. Keybase has implemented usable tools and techniques to bootstrap trust from a variety of sources, including social networks like Github and Twitter, all without trusting Keybase. It’s unlikely that an attacker could compromise all of these sources at once. Once you’ve used Keybase to verify our PGP key, then you can verify the signed packages downloaded from our servers, and actually have a higher degree of confidence that the software is authentic than you would get from HTTPS alone.
Non-security Concerns
curl|bash is not pretty, and people have raised some legitimate non-security-related concerns about it:
- If the connection dies mid-line, bash will execute the incomplete line – however, this can be mitigated by wrapping the whole script in a bash function executed on the last line, as we do. (This is actually arguably a security concern if an attacker can induce a disconnect at a chosen point, but in any case it doesn’t apply to our script.)
- Although almost all package managers allow packages to run arbitrary install-time scripts (often as root), it is far more likely that a wholly hand-rolled install script will be sloppy and harm the system. In the case of Sandstorm, we have taken care to avoid interfering with your system in any way – in fact, Sandstorm self-containerizes in its own chroot under /opt/sandstorm, meaning that for the most part it does not touch the rest of your system at all. See comments at the top of the script for more details.
- Installing software not managed by the distro’s package manager can be inconvenient for system administrators, especially those who are deeply familiar with the inner workings of their distro.
On these concerns: We hear you.
For now, we believe that having our own install scripts allows us to iterate faster, compared to maintaining (and testing) half a dozen package formats for different distros. However, once Sandstorm reaches a stable release, we fully intend to offer more traditional packaging choices as well. We still have lots of work to do!
FAQ
Does Sandstorm’s auto-updater verify signatures on updates?
As of this release, yes. (This is in addition to using HTTPS, as it always has.)
What happens if the release-signing private key is compromised?
We can rotate the key at any time. Certificates on the old key expire in about a month. The auto-updater will begin using the new key as soon as it has installed an update that is aware of the new key. We sign updates with all old keys as well so that old Sandstorm installs continue to be able to update even after key rotation.
Are the installer and updater resistant to downgrade attacks?
Yes. The installer will not install a release more than a month old. The updater will never “downgrade” to an older version. We also plan to add an alert to the Sandstorm UI if the Sandstorm build is more than a month old, which could indicate that an attacker is blocking access to the update server.
Sandstorm Oasis hosting open beta and App Market launch
By Kenton Varda - 31 Aug 2015
As you know, Sandstorm’s mission is to bring open source and indie web apps to a wider audience. To run open source web apps, you need your own server – the developers aren’t a big corporation with resources to run servers for you. And for everyone to run open source web apps, everyone needs control of their own server. That means that running your own server and installing apps on it needs to be so easy that everyone can do it. Most people do not have the time or expertise to edit config files or use SSH, so we need to eliminate the need. Installing apps on Sandstorm needs to be as easy as installing them on your phone – and it needs to be secure by default.
We’ve made a lot of progress on these fronts – try our online demo to see for yourself. However, up until now, to use Sandstorm, you still had to have a Linux machine to install it on. For many people, that’s great – running on your own machine provides the ultimate in privacy. But running a physical machine is still a burden no matter how easy the software may be, and a lot of people don’t want to do it. That falls short of our goal: if we want everyone to be able to use open source apps, we need to give them another option.
Managed Hosting
Sandstorm Oasis is a managed hosting service for Sandstorm. For $6/mo. (but free during the beta), we will give you a web-accessible Sandstorm server on which you can install any apps you want. Choose from the apps on our app market or package and upload your own. Or better yet, package your apps and submit them to the app market for everyone to use.
When paying $6/mo for Oasis, you get a much better experience than you would installing Sandstorm on a $6/mo virtual machine elsewhere. In addition to the ease of use, Oasis knows how to give your apps resources when you are actually using them while not wasting any resources when you aren’t. In contrast, a personal VM will typically reserve a constant amount of RAM for the entire month while sitting idle most of the time. This means that on Oasis, you get the effect of having a server with many times more RAM for the same price.
Some might ask: Doesn’t managed hosting go against Sandstorm’s goal of decentralization? We don’t think so: the most difficult part of decentralization is separating the developers from the hosts. Once you are using Sandstorm apps, you can easily move your data between managed hosting and your own personal server, because the same apps are available everywhere. Sandstorm will always be available to run on your own machine as open source software, so the choice is yours. Moreover, we expect that some day there will be many competing Sandstorm hosting services located in many countries.
Updates for Self-hosters
Today’s launch includes a HUGE update to our user interface. So much has changed that it would be futile to try to describe it; you should just try it. We have, of course, pushed all these changes to our self-hosted users too. Some of you have even been following our Github repo and IRC and helping us test these features early – thanks! For everyone else, run sudo sandstorm update to update now, or wait for the auto-update by the end of today. Don’t have a server yet? Install Sandstorm on Linux now »
Also New: The App Market
Along with today’s launch, we have deployed our new App Market, full of Free and Open Source apps, to replace our old ad-hoc app list. You can now browse apps more easily, complete with ratings and reviews. More importantly, submitting your own apps is now more streamlined: you can upload your package directly from the command-line using the spk publish command.
Our Terms and Privacy Policy
People don’t usually talk about Terms of Service and Privacy Policies. It’s always the same story: a wall of inscrutable legalese that inevitably gives the service provider free reign to do whatever they want, which most users don’t even bother to read.
We wanted Oasis to be a little different. We wanted our terms to address a number of shortcomings we see in “cloud services” today, in each case protecting you, not us. And we wanted to make sure that normal people can read and understand how our terms protect them.
In particular:
- We prohibit ourselves from looking at your data except under very specific, unusual circumstances (such as court orders).
- We promise to notify you if your data is the target of a court order (if we legally can do so).
- We think that protecting your data is the most important thing we do, and we will not delete it against your wishes unless we have no choice. In contrast, many cloud services are known for deleting people’s data on a whim when they violate obscure rules they may not have even known about.
- We do not tolerate harassment. Luckily, Sandstorm’s sharing model provides an elegant way to deal with it, without deleting your data.
- Unlike some people, we explicitly permit white hat security research, so long as no one is harmed and bugs are responsibly disclosed.
- We have zero interest in “tracking” you to learn your interests and build a personal advertising profile, therefore our privacy policy prohibits us from doing so.
Check out the full Terms of Service and Privacy Policy for details – and if you see anything wrong, feel free to file a bug!
Free During Beta
Oasis is not done yet – we have lots to do. We anticipate it will remain in “beta” for several months – and during that time, the service is completely free. To get started with Oasis, head over to oasis.sandstorm.io.
PS: The Name
A couple months back, we asked you on Twitter what our managed hosting service should be called. Nearly everyone who replied – and there were a lot of you – had the same answer: “Oasis”. So, thank you for making the decision easy!
Recap of our August meetup
By Asheesh Laroia - 11 Aug 2015
Last Thursday, RethinkDB co-hosted the first Sandstorm meetup in the South Bay Here’s the view of the crowd at their Mountain View office.
I gave a talk introducing Sandstorm, inspired by Kenton’s blog post explaining the motivation for Sandstorm: making it possible for open source and indie developers to build successful web apps.
Jorge Silva followed that with a talk focusing on RethinkDB. He explained how the open source RethinkDB database makes real-time apps easy to build, and showed how to package RethinkDB apps for Sandstorm. He used the new Sandstorm packaging tool called vagrant-spk.
We then heard four lightning talks.
- Paul Pelzl explained how and why he wrote an implementation of Cap’n Proto in OCaml.
- Warren Togami talked about the importance of security in enterprise chat.
- Andy Lutomirski discussed recent Linux security vulnerabilities, the history of Linux’s surprisingly large attack surface, and his excitement that Sandstorm will bring object-capability security to the world.
- Ross Light showed us the simple file uploading and sharing app he is building using Go. The app looks sharp and he expects it’s almost ready for release. He remarked that it has a 3MB package file, and that could shrink further as he works on implementing replacing sandstorm-http-bridge with new code using his support for Cap’n Proto RPC in Go.
My personal thanks go out to Christina Keelan for handling the logistics and planning that made the event a success.
I plan to organize September’s event in San Francisco and return to the South Bay in October. If you want to hear about those events, join the Sandstorm Meetup group!
Thu Aug 6: First Sandstorm meetup in the South Bay
By Asheesh Laroia - 22 Jul 2015
On Thursday, August 6, the Sandstorm meetup group has its first event in the South Bay. RSVP here!
RethinkDB is graciously offering space and helping provide food and drink. They’ll also be participating by showing off their open-source database for the realtime web and how to use it in a Sandstorm app.
RethinkDB is based in Mountain View, which means easy travel if you’re based in Silicon Valley. After this event, I aim to switch monthly between organizing events in San Francisco and the South Bay.
- Date & time: Thursday, August 6, 6:30-9:30
- Food & drink: provided, with thanks to RethinkDB
- Location: RethinkDB, 156 E Dana St, Mountain View, CA
- Sign up: On the Meetup.com event
One more thing: You should give a lightning talk, where you talk for five minutes about a success story, a challenge or unresolved issue you’ve run into, or whatever else about Sandstorm strikes your fancy. It’ll be easy, and we’re a friendly bunch. Leave a comment on the Meetup event with your idea for a talk!