Dan Luu
patreon
Files, the talk
🕑 Added 2019-06-12 08:59:59 +0000 UTCI'm reading all of the internet comments on an old blog post I wrote on why files are difficult to use correctly (https://danluu.com/file-consistency/) and AFAICT literally every single comment that says something like "you just can do X with files and it will work" is wrong. I've only read maybe 700 comments at this point and haven't gotten to all of them, so I may find a correct comment eventually, but I don't think I've found one yet.
This is unfortunate. I was hoping to address the major remaining misconceptions in an upcoming talk but there are too many of them to really address since it's only a 25m talk. I tend to view misconceptions like this is my fault, in that they wouldn't exist if I explained things better, so I'd like the fix this if at all possible, but I'm not sure how.
One thing that I think is interesting is that, apparently, there are a lot of devs who have One Weird Trick for writing to files that they're sure works. In some cases, I can see how they got there (e.g., a lot of people seem to think that rename is atomic on crash, probably because it's POSIX guaranteed to be atomic during normal operation, but the behavior actually isn't defined on crash, you can't rely on this for crash consistency unless you're using a COW FS, and even then you may be relying on an implementation detail and not something spec'd), but in most cases I don't really understand how people decided that their "trick" works (e.g., just make sure you're append only and nothing will go wrong, but appends aren't guaranteed to be atomic nor are they guaranteed to be issued in program order and I don't see why anyone might strongly believe either of these are guaranteed in the general case).
Another thing that I think is interesting is that this post explained in some detail why writing to files is harder than it seems, but this didn't deter tens of people from responding with some variant of "nope, it's actually easy". I could understand this response if they had a response like "this part of the explanation isn't correct, and because of that, this is easier than this post makes it sound", but none of them do that. Instead, many of the comments are of the opposite form, something like "I don't know why this guy is making this so complicated, this is really easy", i.e., "I don't understand the explanation of why this is hard, it's actually easy", which seems like a really strange line of reasoning.
