Tailscale Down® - Unscheduled network downtime to engage in the immediate needs of your meat sack existence. Eat food, speak with other meat sacks, expel food, touch grass!
Nothing personal against tailscale but I hope it drives the point home that leaving the hosting of the key exchange server for your mesh network to an external entity is a little bit ridiculous.
And for some specific somebody, fixing the problem is their whole job. It’s definitely not my whole job. Maybe not even my job at all (if it’s something I just use as part of a personal hobby.)
Cloud providers have enormous economic incentive to recover from outages as fast as possible and can bring many more people to help, often ones who wrote the code and designed the system. I once worked for a state government where the exchange server was down for two weeks.
I had to deal with a state Medicaid system that would go down often. If it crashed after 5pm, it was down until the next morning when someone rebooted the SunOS box. (Yes, they just rebooted the box, and no, in 2014 it was still sunos, not Solaris). Meanwhile, it’s messing up pharmacy authorization for thousand of elderly and low income people in the state ….
with Tailscale Lock you have a lot more control, you can also self-host your coordinator server which is an alternative even mentioned in the service docs[0]
For me tailscale being down just means I can't access things I'm not comfortable exposing publicly outside my home. It isn't a huge deal as I rarely have that need.
Oh this is unlucky. I'm not affected because I use headscale (for my home network), but this is one of those companies I root for because the product is so good. We used to have manually managed Wireguard (one end terminating at Router) to create the similar effect and for my home, this is way nicer.
It's getting to a point where I _want_ to pay for it so I can demand better service from them :)
Just having tailscale ssh is so damn useful when you've got a bunch of servers virtual and physical and co-located. Just get them to the tailnet and ssh via Tailscale. No need to open SSH ports.
Pretty sure that was the case. I heavily use Tailscale at work and have been working steady on multiple VNC connected clients over Tailscale Wireguard tunnels without issue. Just wrapped it up for the day and hit the ‘ol watering hole (hackernews) to see this. I didn’t connect/disconnect or have to use the portal during that time period, but my in place connections were fine.
Both the web interface and connections are down now. Was just fiddling with my private DNS serving both my local network and tailnet, when I lost access to the device listing around 20 minutes ago. Now I also cannot connect through tailnet IPs anymore. EDIT: took me a good minute to realise this too, as I thought I'd just screwed up something in my Pi's ipv6 configuration for the 3rd time in an hour.
P.S. you beat me to posting by 1 minute! Happy to see there was an option to hide my submission, never needed that before.
Would love to see something like Tailscale but local-first, where your devices use Bluetooth, mDNS, NFC, QR codes, etc for signaling and only fall back to STUN/TURN as a last resort.
Tailscale is definitely not one of that crowd. Their CEO had some very reasonable takes on AI and developers on LinkedIn / their blog (linked in a sibling comment).
Thought this was for a product release called Down. Might be time to go outside for me
Had to have a double take when they released Tailscale Drive. Almost thought I could add my car to my tailnet now.
Totally appropriate if your Tailnet name is goose-rappel.
I'll have to tell my ducks about this one
Tailscale Down® - Unscheduled network downtime to engage in the immediate needs of your meat sack existence. Eat food, speak with other meat sacks, expel food, touch grass!
Tailscale Down™ - What goes up must come...
fade out
Uncertainty as a Service®
Nothing personal against tailscale but I hope it drives the point home that leaving the hosting of the key exchange server for your mesh network to an external entity is a little bit ridiculous.
My self hosted stuff has outages too. The even more annoying part is that it stays broken until I get around to fixing it.
Imagine if stays broken until somebody else gets around to fixing it
There are more somebody elses than there are mes.
Sure would be a shame if they all got together and decided some things for you. But that would never happen. Right?
And for some specific somebody, fixing the problem is their whole job. It’s definitely not my whole job. Maybe not even my job at all (if it’s something I just use as part of a personal hobby.)
Craftsman guilds don’t scale, alas.
I heat my house with a wood stove, I'm not part of the energy grid.
I run my own WireGuard because I like to be the one responsible.
To each their own, no need for snark comments.
If your key exchange server is on us-east-1 are you also ridiculous?
Cloud providers have enormous economic incentive to recover from outages as fast as possible and can bring many more people to help, often ones who wrote the code and designed the system. I once worked for a state government where the exchange server was down for two weeks.
I had to deal with a state Medicaid system that would go down often. If it crashed after 5pm, it was down until the next morning when someone rebooted the SunOS box. (Yes, they just rebooted the box, and no, in 2014 it was still sunos, not Solaris). Meanwhile, it’s messing up pharmacy authorization for thousand of elderly and low income people in the state ….
with Tailscale Lock you have a lot more control, you can also self-host your coordinator server which is an alternative even mentioned in the service docs[0]
[0] https://tailscale.com/kb/1226/tailnet-lock
Are self-hosted solutions impervious to outages? I wasn't aware.
Who said that
"sureglymop" certainly implied it, otherwise I don't know what the point of that comment was.
As LeoPanthera said, it is clearly implied.
For me tailscale being down just means I can't access things I'm not comfortable exposing publicly outside my home. It isn't a huge deal as I rarely have that need.
And if I am at home, the box is upstairs. I can connect a keyboard and monitor and continue about my business.
Just arrived from perusing the headscale repo for the nth time. I really should deploy one of these days...
https://github.com/juanfont/headscale
Oh this is unlucky. I'm not affected because I use headscale (for my home network), but this is one of those companies I root for because the product is so good. We used to have manually managed Wireguard (one end terminating at Router) to create the similar effect and for my home, this is way nicer.
It's getting to a point where I _want_ to pay for it so I can demand better service from them :)
Just having tailscale ssh is so damn useful when you've got a bunch of servers virtual and physical and co-located. Just get them to the tailnet and ssh via Tailscale. No need to open SSH ports.
AFAIK, only new connections can't be established, the already connected clients will continue to work.
Pretty sure that was the case. I heavily use Tailscale at work and have been working steady on multiple VNC connected clients over Tailscale Wireguard tunnels without issue. Just wrapped it up for the day and hit the ‘ol watering hole (hackernews) to see this. I didn’t connect/disconnect or have to use the portal during that time period, but my in place connections were fine.
Both the web interface and connections are down now. Was just fiddling with my private DNS serving both my local network and tailnet, when I lost access to the device listing around 20 minutes ago. Now I also cannot connect through tailnet IPs anymore. EDIT: took me a good minute to realise this too, as I thought I'd just screwed up something in my Pi's ipv6 configuration for the 3rd time in an hour.
P.S. you beat me to posting by 1 minute! Happy to see there was an option to hide my submission, never needed that before.
hide just removes the submission from your page/new, it doesn't remove it from HN.. delete does that.
Good to know, but then again, I don't see such an option for my submission. Guess I'll just have to live with it.
You can only delete a submission in the first 10 minutes and only if nobody has commented yet.
Connections through tailnet are back, website is still down
Website is also back, nice recovery speed by tailscale
Would love to see something like Tailscale but local-first, where your devices use Bluetooth, mDNS, NFC, QR codes, etc for signaling and only fall back to STUN/TURN as a last resort.
Magic DNS has been broken on Android for weeks, ever since One UI 8.0 was released. No monitoring dashboard for features AFAICT. Bummer.
For example, see: https://old.reddit.com/r/Tailscale/comments/1onqe58/android_...
I'm surprised TS Team doesn't monitor it for issues.
<3
I was going crazy trying to figure out why I couldn't connect to our supercomputer an hour ago. Relieved to know it was a global outage!
As someone pointed out recently here: no one gets fired for AWS outage
Are you replying to me?
Good luck tailscale, I love how much value I get out of your free tier for my home servers!
Massive outages of core Internet services popping up shortly after corps firing devs and bragging about AI writing more of their code.
I could be mistaken but I don't recall Tailscale being one of these?
Tailscale is definitely not one of that crowd. Their CEO had some very reasonable takes on AI and developers on LinkedIn / their blog (linked in a sibling comment).
But Tailscale IS VC funded which means an exit is imminent and around the corner.
Not good.
Tailscale isn't a massive corp, more like a Series B startup. And the CEO's take on LLMs is a sober one, not based on hype.
https://tailscale.com/blog/ai-changes-developers
I’m sorry, Apenwarr is the Tailscale ceo?
Weird how you notice a few names on a message board then they disappear to do something new.
Self host with Headscale.
My Headscale instance is working perfectly. Glad I didn't just use Tailscale for my homelab.
Lemme guess - us-east-1?
Looking forward to the post-mortem.
Yep... cant wait for them 'unwrap' the reason.