Let’s start by stating something I believe is so obvious it shouldn’t need stated:

You should not have to worry about your tools spying on you.


You should be able to run a command that doesn’t use the network, knowing that it won’t open a network port. You should be confident that your tool is doing its best for you, not reporting back on you to someone else. In short, you should be able to run software without it looking over your shoulder like a voyeur with a clipboard.

But nearly a year ago that kind of spyware is just what Microsoft/.Net Foundation added to the dotnet command line.

I’ve been using the dotnet core since well before then and I never knew about this. And I’m one of the few people I know who tries to keep up with this kind of nonsense! I feel foolish and embarrassed for not knowing about this spyware when it was added. And maybe my embarrassment at having been spied upon for months is colouring my judgement a little. But I still believe it’s wrong.

I’m sure they’ll say that it’s to improve the tools, but - while I have my doubts that’s true - it does bring up the question:

Would you prefer a tool you can just trust, or a tool that may have better features but that you constantly have to check to verify isn’t doing anything it shouldn’t?

I’d rather be able to trust my tools. I just don’t like the idea of a voyeur with a clipboard watching over my shoulder, sating its prurient interest by taking notes and gathering statistics.

This dotnet voyeur then sends these notes and statistics to Microsoft without asking the user.

Your only chance of opting out is knowing the special environment variable incantation to use.

But maybe they’ve tweaked it so that today it’s sending files as well? They managed to sneak the first change past me, so have I missed another? No? Maybe not. But tomorrow? I can’t know, since they’ve demonstrated I can’t trust them or the tool they created.

What used to be a simple ‘dotnet run’ command has turned into something that has me watching my back. Why are they so interested in my typos that they’ve paid someone to sit down and write code to capture them? If they actually want to improve the product, why not have that developer writing code that adds new features rather than spying on me?

And that’s why it’s not a minor thing. I’m not (quite) so arrogant that I think Microsoft is targeting me. I don’t even think they’re especially interested in the telemetry from ‘dotnet run’. It’s that they’re seeking to normalise this spying that makes it more than a minor problem.

We’ve seen this with Windows 10 hoovering up all the data it can get, just like Facebook, Google, Apple and Amazon. It’s in all their interests to have us become inured to this constant surveillance. And I don’t like it.

Homebrew faced a similar issue around the same time dotnet introduced their telemetry. I noticed the Homebrew debacle but didn’t notice the introduction of telemetry in the tool I use all the time. (I’m still embarrassed by that.) To show I’m not the only person concerned about telemetry-gathering tools, here’s a blog post about Homebrew - ‘Homebrew betrayed us all to Google’. It starts with the summary:

    1. Open-source is about trust. Trust is underminded by things like tracking.
    2. Do not track your users. In the rare case you really need anonymous data, ask your users first.
    3. Never use Google products (or any other “big data” company that relies on making money out of the data you provide) to track your users.
    4. Using Google’s tracking and then calling it “anonymous” is a lie. Google collects tons of information of its users and even non-users. There’s no way to know what data Google will relate internally. Even if you don’t get to see all of the collected information, Google still has them.
    5. Opt-out is never an excuse. It always excludes most users (which either don’t care, or have more severe things to care about than protecting their privacy in every random app they’re using).

(Source: ‘Homebrew betrayed us all to Google’)

Homebrew backed down a little and provided a better opt-out mechanism, but it annoyed a lot of people. (More, probably, than are annoyed at Microsoft. Let’s hear it for low expectations!)

Opt-out mechanisms aren’t really enough though. For one thing, why should I have to opt out when I didn’t opt in in the first place? For another, that may fix it for me, but I don’t want your tools spying on you either. For a third, the opt-out procedure is (deliberately?) awkward.

It’s not something you just pick, it’s something that needs to be set for every user on every machine in every shell and every container. And you need to get it perfect every single time, or else the tool will assume it can report back on what you’re doing.

Opting everyone in automatically as Microsoft have done is just plain dishonest. There’ll always be some portion of users who’d opt in, some portion who’d opt out, and some who’d go with the default. But you know what, Microsoft? Those people who wouldn’t have opted in but who haven’t opted out? They’re the ones whose data you’re taking without permission. You just don’t have permission to take that data. (Don’t start me on EULAs when the person agreeing to the EULA may not be the person running the software…) You don’t have informed consent here, because you didn’t actually ask. Worse - you know that if you asked for informed consent, you might not get it. That’s an argument against spying on people, not an argument for spying and not asking.

And that’s before you get to people like me who - despite what you consider ‘transparency’ - didn’t even know there was a possibility of a voyeur with a clipboard looking over my shoulder.

So how could Microsoft fix the issue?

There’s really only one fix I’d like - take the spying code out of the tool completely. If there are people who really want to send their telemetry to Microsoft, by all means find a way to accommodate them. But don’t put spying code into the tool. Keep it clean. Have the telemetry spyware in a separate module that has to be explicitly downloaded and installed. (Call it ‘Voyeur.DLL’ if you like.) Keep the core pure.

And have a strong ‘Private By Default’ policy. Allow people to feel safe using your tools. It’s hard enough keeping up with the latest in technology without having to keep up with the latest in obnoxious business practices.

Private By Default would mean guaranteeing that it never gathered any information on you, even in aggregate. That it never sent any data that you didn’t explicitly ask it to send. That it never opened any network connections you didn’t ask it to open. That it never did anything not explicitly to do with carrying out the user’s intent.

In absence of that, what can I do to stop it spying on me?

  1. I could just not use dotnet. For me this is the easiest and the hardest approach. It’d be easy because just walking away from dotnet would mean it’s not my problem any more. There’d be no voyeur looking over my shoulder. It’d be hard for me too though. I’m getting to the point where a large side project is becoming useful, and it’s based on dotnet. It’d be difficult just to walk away from that.
  2. I could block telemetry traffic on the router or firewall. Here’s someone’s (not my) best guess at the hosts to which it sends data. I like the idea of ISPs blocking all those hosts - denying access to login.windows.net because of Microsoft’s telemetry-gathering could be hilarious.
  3. I could wrap the dotnet command in a script that automatically sets the environment variable for every single invocation of the command. Here’s one way to do it:

    #!/bin/sh
    echo "Trying to run a non-spying version of dotnet..."
    DOTNET_CLI_TELEMETRY_OPTOUT=true /usr/local/share/dotnet/dotnet $*
    
    (That’s for bash on OS X - if you call it ‘dotnet’, make sure it’s on your $PATH ahead of /usr/local/share/dotnet/dotnet.)
  4. I could add the environment variable to every single RC file for every single shell for every single user. And every single docker file. For every single development machine and server.

I’ll be doing a combination of all those things. I might keep using dotnet for existing projects, but I’m fucked if I’m starting any new dotnet projects now.

The ‘tech stack’ conversation has come up in $WORK a few times recently. Where before I’d have talked about dotnet core I’m sure as hell not going to now. I won’t just not be talking it up, I’ll be actively talking it down and discussing alternatives.

From a wider perspective, what could I do to fix the root of the dotnet spying problem?

  1. Rewrite the part of the tool that calls the spying code. It’d be easy enough for me to fix (it’s right here), but that wouldn’t solve the problem of Microsoft writing tools that spy on users, it would just stop my version of the tool from spying on me. Your version could still spy on you.
  2. Send the code change to Microsoft as a ‘pull request’. I think we both know what would happen with that.
  3. ‘Fork’ the code, and provide a binary distribution of the fixed/improved code so that everyone that wants can use it.
  4. Start a ‘Private By Default’ campaign in the hope we can shame Microsoft into behaving better.

But you know what? I’m not going to do any of that. I’m just going to point out why I think it’s wrong, then try moving on to using better, more trustworthy tools. I’ll still use it for current projects but I’ll be trying to move away from the platform.

Today I was planning on settling down to read the new AssemblyLoadContext design document pull request and delving a lot deeper into that area. My dotnet project needs to generate and load assemblies in different contexts and it has got as far as it can without this kind of functionality. I might even have written a blog post about it. After all, it’s an area not well served by others and the documentation doesn’t go into a lot of detail about how to use the API.

Instead I’m writing about how dotnet has managed to shatter my trust.

I’ve no enthusiasm for working with dotnet now. No desire to watch the weekly ASP.Net standups. No desire to write C#. No desire to work on my side project built on dotnet core MVC. I keep looking around for the voyeur with the clipboard.

Tags: Clueless Idiocy
Created by on Logo15659OpinionatedGeek Ltd.Logo15659

‘Excellent Shift Of Viewpoint’

Score: 5/5

Charles Stross
£8.99

The Laundry series was fun - a bureaucratic civil service department tasked with countering the unspeakably evil magic everywhere and making sure it didn’t become public knowledge. It was maybe showing its age though and getting a little tired.

This book kicks things into a different gear. As well as seeing the beginnings of major changes in the overall story arc (I'm trying hard not to mention anything spoilerific...) this book is told from a different character viewpoint. Instead of Bob being the narrator, it’s Mo.

That one shift leads to a big change in perspective as well as taking the plot in an entirely different direction. My empathy with the character seemed higher (as did the frequency of my saying ‘No, don’t do that...’ to her in my head) and she seemed a more fully-rounded person than Bob.

Fun to read. I’m looking forward to the next instalment now.

Tags: 4 Word Book Reviews
Created by on Logo15659OpinionatedGeek Ltd.Logo15659

‘Fine Romp Through Science’

Score: 4/5

Marcus du Sautoy
£16.59

Marcus du Sautoy was over in Belfast for our Science Festival and SWMBO and I got to see him give his talk based on this book. It was thoroughly engaging so of course she bought me the book!

The book covers far, far more than he could mention in his talk - he only really talked about 3 of the ‘edges’ out of the 7 in the book. What he did cover was interesting though.

On the other hand, I did say to SWMBO we could gauge how deep a talk it would be by noting when he mentioned Gödel’s Incompleteness Theorem. Gödel’s proof that there are true things in mathematics (or really any formal axiomatic system) that you cannot prove are true is an obvious candidate to cover when talking about the limits of knowledge.

Or so I thought, anyway.

Sadly, Gödel didn’t crop up until the questions at the end. Ah well.

It does get talked about in the book though. The book covers so many topics that Gödel’s incompleteness theorems aren’t covered in any great depth, but they are there and covered well. (And as a side note, this reminds me how remarkable Gödel, Escher, Bach was when I read it decades ago. I have an urge to read it again, but not at £19 for the paperback! I may hunt down a secondhand copy...)

I’m a programmer though (no kidding!) so one thing I’m really disappointed that didn’t get a mention in the book is the Halting Problem.

What is the Halting Problem I hear you cry?

The problem is to determine, given a program and an input to the program, whether the program will eventually halt when run with that input. In this abstract framework, there are no resource limitations on the amount of memory or time required for the program's execution; it can take arbitrarily long, and use arbitrarily as much storage space, before halting. The question is simply whether the given program will ever halt on a particular input.

And in 1936, Turing proved that sometimes you just couldn’t know:

Turing proved no algorithm exists that always correctly decides whether, for a given arbitrary program and input, the program halts when run with that input. The essence of Turing's proof is that any such algorithm can be made to contradict itself and therefore cannot be correct.

This well-known thing-you-cannot-know seemed like such an obvious candidate for a book on Things We Cannot Know that I’m genuinely surprised it doesn't make the cut. Turing gets 4 mentions in the index, but they’re all about the Turing Test rather than this.

That quibble aside, the tour of current science in the book does cover topics like chaos, quantum mechanics, relativity, time, consciousness. And my copy of the book is signed by the man himself. No, you can’t have it.

Tags: 4 Word Book Reviews
Created by on Logo15659OpinionatedGeek Ltd.Logo15659

‘More Big, Big Ideas’

Score: 5/5

Cixin Liu
£6.29

This book was truly remarkable. It’s one of those books I want to tell everyone who is interested in science fiction to read. If you like the science-heavy (and perhaps character-light) science fiction of Arthur C. Clarke, I think you’ll like this.

You do need to start with The Three Body Problem, then The Dark Forest, but it’s well worth it.

The annoying translator from the first book is back, and he once again feels the need to litter his translation with footnotes. These footnotes really do break the flow of the book. I preferred the second book’s approach - it was translated by someone else.

But even with the annoying and sometimes klunky translation, this is an incredibly thought-provoking book. It’s packed with ideas, including some of the Big Ideas from current science. Some of the ideas are questionable - I did find myself saying ‘If they could do X, why didn’t they do Y...’ a bit - the applications of some technologies seems maybe a bit inconsistent.

Even so, they’re minor quibbles about an enjoyable blast through future possibilities.

Tags: 4 Word Book Reviews
Created by on Logo15659OpinionatedGeek Ltd.Logo15659

‘Paradox Free Time Travel?’

Score: 5/5

Edward Aubry
£14.99

(Side note: look, I know the last lot of books have all been 5/5. The next few are as well. They've all been remarkably good and I’ve thoroughly enjoyed them. There’s also the inherent bias in that I’ve generally stopped bothering with books I don’t like, so yeah, there’ll be a lot of 4/5s and 5/5s. Still, if your taste matches mine, books with these scores are well worth reading. And if you don’t like seeing so many 5/5s, if you’d prefer I read more piles of wank like ‘Coalescent’, tough!)

Time travel stories must be difficult to write. It’s pretty easy to point out flaws, and most have obvious problems with paradoxes. It’s how they tackle the paradoxes that can be interesting. (And let’s all agree that the Back To The Future fading photograph is a bad approach, yes?)

This book has an interesting take on the paradoxes. I’m not going to go into details (no spoilers!) but I’m heartened that it’s at least addressed.

The story itself is interesting, and the sequences of events in and out of order gives a good perspective for the character.

Tags: 4 Word Book Reviews
Created by on Logo15659OpinionatedGeek Ltd.Logo15659

‘Series Now Consistently Good’

Score: 5/5

Ben Aaronovitch
£6.99

The previous book in the series got 5/5 too, and I think this series has reached a good level now. There’s a good balance between the individual story and the overall story arc that it’s worth reading each story on its own merit as well as wondering what’s going to happen next in the series.

I do wish he’d get over his obsession with telling us all the architectural details though.

Tags: 4 Word Book Reviews
Created by on Logo15659OpinionatedGeek Ltd.Logo15659

‘First Locked Room Mystery!

Score: 5/5

Israel Zangwill
£2.57

Before Death In Paradise, before Jonathan Creek, there was this - the first ever ‘locked room’ mystery.

I do like a good puzzle, and something about this particular kind of puzzle appeals to me. SWMBO knows this, so bought me a copy of The Big Bow Mystery for Christmas. It’s out of copyright now, so it’s quite cheap to buy but you can also just legally download it from Project Gutenberg.

The setting could take a bit of getting used to - it’s set, naturally enough, in the 1890s since that’s when it was written. If you’ve read enough Sherlock Holmes (also available free on Project Gutenberg...) you’ll be familiar enough with the language and idioms though.

Did I figure out whodunnit? Nah. By the close of the book I had a few scenarios in my head and one of them was right, but I hadn’t spotted enough clues and discarded enough red herrings to narrow it down. The art of the whodunnit seems to be to give just enough clues to allow things to be figured out in hindsight, while adding a whole heap of misdirection so that the reader can’t sort out what’s important and what isn’t. There’s plenty of misdirecting fun here.

Tags: 4 Word Book Reviews
Created by on Logo15659OpinionatedGeek Ltd.Logo15659

‘Diversity In Science Fiction’

Score: 5/5

Nnedi Okorafor
£7.99

This is a short book - I didn’t know it was a novella when I ordered it so I was surprised when it turned up at less than 100 pages. £7.99 for a novella? I must be getting old.

Anyway, I enjoyed the story. I think at its core it's about diversity and alienation, and it captures those problems well. The writing does a good job showing the effects of being in the ‘out group’. The science fiction aspects are maybe a little ropier, the politics and organisations more simplistic, but that still took a back seat and I was still cheering for the main character through it all.

Tags: 4 Word Book Reviews
Created by on Logo15659OpinionatedGeek Ltd.Logo15659

‘Rationality And Cognitive Biases’

Score: 5/5

Daniel Kahneman
£7.69

I finally finished this book! It took me a long time.

I started this before we went to New Zealand in 2015. I decided not to bring it on holiday there, and that pause made it difficult to get back into it. I still picked it up every now and then, but it never really grabbed me.

That’s a shame because it really is a good book. Evidence-backed descriptions of irrationality and biases in common patterns of thinking, detailed by the man who won a Nobel prize for the research.

It’s been a few years since publication though, and it turns out there are a few problems with the book contents. It’s still a remarkable book, and worth reading. Just don’t take as long over it as I took.

Tags: 4 Word Book Reviews
Created by on Logo15659OpinionatedGeek Ltd.Logo15659

‘Forecasting Seems Hard Work’

Score: 4/5

Philip Tetlock and Dan Gardner
£6.99

There’s a lot to like in this book. It shows how some folks who have disproportionately good results when forecasting actually go about creating their forecasts.

A lot of it focuses on a similar theme to Thinking, Fast And Slow, a book I started over a year before I started this book but I still managed to finish this one first. Maybe that says something about their relative readability. Or my ability to stick with things.

Anyway, this book also discusses rationality and biases, and how particular people in particular circumstances have ways to overcome those biases. It provides the basis of a toolbox for the reader to follow along and learn to overcome their own forecasting biases.

Tags: 4 Word Book Reviews
Created by on Logo15659OpinionatedGeek Ltd.Logo15659