> But it is really hard to ignore your own biases. Of course you know that certain commands require sudo and obviously when you wrote -foo you meant --foo and everyone knows that you have to reboot afterwards.
I sometimes write readmes for myself, so that I can remember the exact steps to generate a data report, etc.
It's surprising how much they become incomprehensible after just a couple of weeks; when everything's in our head it's all clear, fluid and self-explanatory; but once we have forgotten the context, nothing makes sense anymore.
Most people don't think linguistically, we have way more conceptual thinking. What makes it difficult is we rarely realize we are thinking conceptually when writing because these concepts occur automatically and we don't notice it.
In school this lead to a lot of difficulties for me, one in writing those comments out for other people to understand, but the other seemed to be that when reading the average statements used in education for teaching I could map the same strings of language to multiple and sometimes conflicting statements because of the inexactness of the language used.
It turns out explaining concepts while leaving little room for different interpretation is hard.
Also - formalize what can be. There may be e.g. opportunities to write scripts that can be tested and linted. I realize this isn't always possible, but I do think it should be ruled out rather than ignored.
>I hadn't actually explained what the software would do.
Half the posts I see lately are like, "Gleam 2.0. What we learned" and then you go to the homepage and it's "Gleam is a Tribble for your Fork! (Scroll down) See if you qualify for Gleam Enterprise!"
I was confused by your mention of "Gleam", but I suppose it is just a generic name, not referring to Gleam [0], the functional programming language, as the latter is an example of a project that focuses on concise and minimalist documentation (btw, its README is just a reference to the website).
I think I read somewhere (and in any case it matches my experience) that the first user will catch 50% of the problems and that with 5 users you can catch close to 90%. So human testing is not just invaluable, it's pretty cheap for the benefits it brings.
You want to hire your target audience, which may be vastly different from yourself, and suddenly you discover that the language is throwing them off, the color scheme brings different meanings, and they just don't understand the flow which felt completely natural for you.
We most times forget that not everyone can read our minds or see exactly what we see in our systems. When it comes to communicating ideas, there's always the requirement to actually communicate what it is the readers needs to know.
I oftentimes find myself spending more time rewriting readmes than writing code.
Treat the readme like a journey/walkthrough of your product, follow an order, and keep it simple to understand.
I love this. Most readmes are plain bad. I think the most egregious is when a readme doesn't state what the project does. I get that not every project is aimed at the public, but if you go through the bother of creating a readme file, why not go the extra 10 centimeters by writing the most basic information?
Other issues:
* outdated (and thereby wrong) information
* using un-introduced abbreviations (bonus points for abbreviations that have common meanings, e.g.: EG, NB, IE, ETC)
> My jokes aren't funny and are actively confusing.
I used to write technical documents in prose style, sometimes with meandering stories. I guess I picked it up from my early blogging days. I realized I hated reading some of them back. So I tried to keep it cut and dried. I do sometimes sprinkle a bit of colorful wording just to add a bit of humanity but only if it doesn't get in the way of the main message.
When deciding on a style for documentation, I typically draw the line between tutorials and references. The linear top-down flow of an introductory guide lends itself well to inserting additional context throughout it even if not completely on-topic, while in an API or hardware reference you generally want to keep each section reasonably self-contained, trivially searchable for (minimizing false hits by carefully choosing keywords) and readable independently of the others. I have found the literate programming approach [1] of writing entire tutorials as code to work pretty well for this purpose, which I've used to great effect in some of my pet projects [2].
Being short and concise is usually the better way.
I tend to be too verbose in writing as I want to explain the context in more detail, but short, accurate, concise is best. Many developers fail at that too, though. Many projects do not have working examples. That annoys me the most. It sends a message of "I don't care about new users learning how to use my project".
Same for me. What helped is the realization that I was trying to cater to everyone in the same document. Now I try to follow the organization outlined in https://diataxis.fr/ I’m still very bad at documentation in general but I’m less dissatisfied when I come back a few months later.
>"I don't care about new users learning how to use my project"
I published something the other day with minimal instructions, and felt briefly conflicted.
But I figured, if you want to run it, you'll find a way! (It probably doesn't even work on other operating systems, but porting it would take what, 20 seconds of Codexing?) It was true before AI, and it's definitely true now.
My intended audience is people who want to get their hands dirty. Though I suppose these days, that's the machine's job...
Documentation is so important. I have been treating documentation in the same way I handle code. I found mkdocs works pretty well and integrated with a github job that updates the docs when I commit changes to my main branch. It also allows contributors to correct errors or add helpful instructions to documents. I think I have not paid enough attention to my biases though and like the idea of hiring someone to go through the process. I think my instructions are sound but maybe not so much for a user who is not as familiar as I am. I may not be doing things the optimal way but I have used a lot of documentation over the years and feel what I have done addresses gripes I have had with "Big Tech" provided docs.
mkdocs still uses "web fonts" for icons. This hack was required for compatibility with Internet Explorer, but is just cargo-culting these days. It's an accessibility issue in that users who disable web fonts (for readability) don't see icons, they just see unicode placeholders everywhere ("tofu").
This sounds exactly like UX usability testing. Sit down with someone and watch them work through whatever process you’ve created.
It’s normal to compensate them for their time.
Normally though you don’t modify it after each participant. But for something very niche like following a README (as opposed to an e-commerce flow targeted to the general population) it might be fine, if less rigorous.
There's a zeroth step missing from both the Quickstart and Full Set Up: install php-fpm, configure it, and configure your http server to serve using it as a backend.
It seems to be taken for granted — that's something you'd already have in place if you're already serving applications in PHP, but would have to figure out on your own if you haven't served anything in PHP so far in your life.
It's a tricky problem. How far back in the stack do you go? The README assumes that you know how to use git to check out the files - or that you can easily save them from the repository. Should it include that as a step in the tutorial?
As I say in the linked article, it depends on what sort of user you have. For a "getting started with Raspberry Pi" document, you might well want to include how to insert an SD card etc.
I'll have a think about the best way to help people figure out if they're running PHP. Thanks for the feedback!
Feedback and issues you need to troubleshoot with your projects is a good indicator of your audience level, and from my experience it’s helpful to understand that documentation is always under development just like the code
from experience in ENT support where i was sending instructions & quick fix scripts to technically capable persons, you will learn very fast when you’ve missed the mark with your documentation / instructions. tons of times i had ready made solutions that i thought “just copy and paste and go what could possibly go wrong?” and was caught off guard how often a little too much knowledge lends to confusion. i am not blaming the users here it is my fault that i didn’t explain things like “no don’t change this date in the fix that is a special date when the issue could have earliest occurred and it’s there to avoid grabbing more than we need to parse”, but i didn’t tell that so of course people changed it to all sorts of dates thinking they had to
such feedback and issues also got me way better about writing code that avoided chances for such mistakes as i didn’t want users to have to read a novel to understand what to do; it’s a fine balance between what to solve with documentation and what to solve with code
Installation instructions usually (should) have a “prerequisites” section. You don’t have to explain how to install the prerequisites, but they should be listed.
My rule of thumb for my READMEs: there should be a list of commands, that when executed in order and in a clean machine, result in the software doing something useful. Yes, this includes `git clone`.
If there's something the user might already have, like the webserver, I add a comment "skip this if you already have a web server". If there are any shortcuts that make it not production-ready, it's time to break out the ALL CAPS.
Limiting the operations to simple commands also helps me keep honest about the instructions (no hidden assumptions), and forces the software to be minimally testable.
If Windows/MacOS doesn't ship git by default, then yes, that should be included. On Linux, the people who are running Linux From Scratch can probably infer what the problem is.
Generally it seems like good READMEs assume you have a compatible OS ready to go, but will give you a summary of all commands to get a working setup from there.
Even as someone working with PHP I would prefer if the project provides we with some guidelines for setup. Especially if it requires some specific version or extension. Ideally the whole dev environment should be containerized. Then you would again require people to understand and use that layer, but depending on the projects complexity definitely something to consider to make it easier to work with
I know there are some great README's (and other documents) around, that document best-practices or templates for great README's. I found one that looked very useful and would've sworn I starred the repo to find it again in time of need. Alas, can't find it. Anyone has some good resources to point to, to add to this thread?
Update: Found some related HN threads (omitted link-rotted submissions).
LLMs are also good for this; point them at your repo and ask them to do a fresh install and note all friction. One time when the lack of context really helps.
Most documentation reads like you should already know what you’re doing, which makes sense because it was written by someone who already knows how to do the process.
I think good technical writing requires the same skills as good product ownership, that is empathy for the user and their perspective. Often technical writing is an after thought and not someone’s whole role and it really shows.
Yes. The amount of projects that don't just run is outstanding. Luckily docker container projects are inherently better at this is in terms of dependencies, but there are still often weird assumptions or medical incantations to get them during.
The link says "My opinion is that FAQs pose a problem only when there’s no strategy around their usage." Calling them "problematic" based on that seems like an exaggeration
"They were the ones who caught the mistakes that no spell chequer could."
Nice, good article lol. I appreciate a joke or two in a readme but totally understand the annoyance, I think forgetting to actually say what the software does is super common as well though. Often trying to figure out if something found on github will actually solve a problem only to be met with a list of install commands.
The 'look awesome project with funky name, here's how to install off you go' without a word to what one earth it actually does is amazingly common. Plenty of things posted here I bounce away from after hitting the linked page, finding something someone is obviously very proud of, but not having a clue what it's actually about :p
> My jokes aren't funny and are actively confusing.
Confession time.
I used to be prone to giving things slightly silly names, particularly unrecoverable structured exceptions. Examples include PancakeLandingException, ReallyBadException, CataclysmicException, ApocalypticDeathException, and the like.
And then years and years ago I used to work for a company called Redgate and I started a tradition of slightly silly messages when early access builds of our .NET products would expire, all based on Monty Python sketches and quotes. So, obviously, the dead parrot sketch featured in there.
So far, so harmless, but this did come to a head somewhat spectacularly and in a couple of different ways.
Firstly, in 2009 one of my colleagues used a modified quote from The Life of Brian as an expiry message on a build. Somebody who appeared to be some sort of religious zealot complained loudly to the company. We were both in LA at the time, with a couple of other colleagues, attending build 2009, so we woke up to a chain of something like 50 panicked emails in our inboxes with people expressing differing levels of outrage and/or amusement whilst discussing various grovelling apology options... until someone figured out that it was actually an elaborate troll that we'd swallowed hook, line, and sinker. I can't remember the name of the person who caught us out but, hats off, well played, sir, well played. It did unfortunately mean we became a bit more cautious and business-like with our early access build expiry messages.
Secondly, and this one needs a bit of context setting... I've always been a fan of descriptive and explicit error messages: there should be enough information in any error message that most of the time the user can figure out what's wrong and fix their own problem OR at least so that if they get in touch with support, then support can quickly figure out the problem and get back to them with a solution. I'm not a fan of unhelpful, information poor, obfuscatory, or cryptic error messages.
But when I wanted to cause an application to exit because there'd been an error related to tampering with our licensing code I played somewhat against type. I wanted error messages that would uniquely identify what had happened, making it easy for us to figure out, whilst giving the user no clue (because I wanted to make it very slightly harder for hackers/crackers - but let's be real: this would never have actually stopped anyone). So I used successive lines of dialogue from a scene in House where House is trying to guess who Wilson's girlfriend is. I have no clear recollection of why I chose this dialogue to reproduce, but... I did.
Anyway, this did lead to some slightly confused support requests coming in from users mostly trying to use the tools legitimately in slightly unusual scenarios, but nothing that was overly burdensome. That was until early 2011, when Greg Young - he of event sourcing fame - posted the following gist because he'd encountered an error that said, "Because I wanna ask you about your girlfriend. I must know who she is, or you would've told me her name.": https://gist.github.com/gregoryyoung/871736.
Not at all creepy, right? And, of course, it went viral on twitter. Cue another massive email thread although, this time round, people just thought it was funny. However, we did decide to make the error messages a bit more boring and, in the end, I just gave them numbers.
Mostly I'm just glad the error Greg got wasn't the final line of dialogue in the exchange between House and Wilson: "Yo mamma." That would have been bad.
Somewhat related question: what is it about READMEs that AI agents love to dump the most useless, hard to contextualise & comprehend, irrelevant rubbish into them that makes understanding a project and onboarding so hard?
It feels like like the AI agents can't help themselves sometimes, and the judgement exercised around what's included and omitted is baffling.
But maybe READMEs have always been this bad, and AI agents have raised the baseline?
That sounds pretty similar to what I get from LLMs regardless of what they're writing for. Any time I get more than a paragraph out from an LLM, it tends to have of a lot of irrelevant details or unnecessary hedging, and I have to either wade through it to find the actual answer or try to convince it to make it more direct.
As someone on the spectrum, this does happen to me with people sometimes too; I struggle when asking a question and getting an answer that doesn't fit the "shape" of what I expect (e.g. asking a yes or no question and getting a relatively long sentence in response that doesn't contain either "yes" or "no" in it, which means I need to do the equivalent of applying it as a diff to my mental model and seeing if there are conflicts). This happens less frequently with other humans though, and on average the amount of effort I need try to figure out what they're saying is a lot lower. This doesn't make it less frustrating when I get that kind of output from an LLM, but it doesn't surprise me all that much that it happens.
I'm sure I do this all the time to people too, though. One of the biggest lessons I've learned in the past several years is that I communicate in ways that I'd probably have trouble understanding in reverse a lot more frequently than I realized, even I still do the things I find confusing from others less often than most people I interact with. I'm sure a lot of people might find this comment to be pretty much exactly like what I'm complaining about even though I feel fairly confident at least in this moment that my point is clear.
> As someone on the spectrum, this does happen to me with people sometimes too; I struggle when asking a question and getting an answer that doesn't fit the "shape" of what I expect (e.g. asking a yes or no question and getting a relatively long sentence in response that doesn't contain either "yes" or "no" in it, which means I need to do the equivalent of applying it as a diff to my mental model and seeing if there are conflicts). This happens less frequently with other humans though, and on average the amount of effort I need try to figure out what they're saying is a lot lower. This doesn't make it less frustrating when I get that kind of output from an LLM, but it doesn't surprise me all that much that it happens.
I've have people in my life where I've got situation going to the point I got sick of it and just started responding 'That didn't answer my question. Do you remember what my question was?'. Turns out many people seem to invent out of whole cloth what your question is rather than actually responding to it. Seems like a whole lot of wasted effort. I mean, it's one thing to ask a yes or no question and getting 'it depends on X, Y, Z, blah blah blah', or just 'I'm not sure/don't know'. There have been far to many cases where I ask something and the response is functionally 'well you see, it all began in 1969, back when dinosaurs ruled the earth'.
LLMs seem to atleast actually respond to your question. The question might be phrased in a way which doesn't match what you intended, and the answer will be verbose and require you to wade through prose to actually find it, or it might just be wrong, but it will at least be an answer to the question you gave it.
Indeed, these make me back out of a project immediately if I don’t have a strong need to use it, and it takes extra cognitive effort to ignore the emojis and focus on the actually meaningful text.
This one wasn't as bad as the majority that have the emoji in each header too. At least these emoji did seem to properly represent the item they're next to, not an immediate rocket ship emoji in the header following by irrelevant ones of various sizes littered throughout the text.
To me it’s similar to what you wrote about the jokes you removed: the emojis are distracting and not actually fun. This use of emojis comes across as a tired meme. It’s better if the project makes people smile due to its intrinsic qualities like working well and doing what they want.
Given the agents.md I’m assuming the readme was just spit out by an LLM and the goal was to not read as LLM text. Problem is, it reads like you prompted it to be written in more simple language.
I just find it a bit misleading that you’re saying you want to talk to real people all while trying to clean up AI slop.
Originally had replied “explain the agents.md then” but decided I should actually check it myself… lol fair enough
I will say, if I did stumble on your project, just seeing that file there would be enough to make me skip past and not spend time reading. I look for those before investing any time reading a readme.
In general, software is still Beta if a program requires a readme file to install and use.
It is under 5 minutes to write a shell/bat/make/cmake script for each platform to configure library requirements, import OS specific data, and enable GPU/NPU features. Then run through the application build, installation package with stripped performance build, and or a few regression tests.
Lets say you have 4k downloads a month on a small project, and it takes 1 hour for each admin to read/configure. You just saved about 5 years of your users lives reading your document. =3
Writing good documentation is difficult. From those who say
"the source code explains everything", I think 80% are too
lazy to write documentation in the first place.
Having said that, I found consistently that when a project
has working examples, ideally documented a bit, aka explained,
they tend to work much better than those projects that have
no examples. Working examples often also help get into a project
quickly and check out how it works. It helps to learn too.
READMEs are not useless, of course, but the quality varies a lot.
I also know of folks who use AI slop spam to improve it, but while
it may improve a little bit, it generates a lot of horribly to read
text that makes no sense. I am noticing this with the ruby core dev
team - they (almost) all suddenly have perfect language skills but it
is more like an advanced babelfish translator. What they piece together
here makes no sense. Claude in particular is now famous for this slop
content. And I don't understand what it is used: real people read any
of this AI slop? Because I just skip it or filter it away these days.
>From those who say "the source code explains everything"
Because theoretically you should be able to describe not only your entire application logic, but upper and lower bounds of inputs as well. Good code would describe this inherently.
Documentation is only useful when a) the application is not source available, so you have no choice, b) you want to save a human developer time for them to understand your code, or c) you want to use documentation as a cache hit for agent use (less token spend)
>But it is really hard to ignore your own biases. Of course you know that certain commands require sudo and obviously when you wrote -foo you meant --foo and everyone knows that you have to reboot afterwards.
For God's sake, this is not "biases," wtf?? That's straight up just not reading/following the document you're supposed to be testing/reviewing. When I test my documents I actually follow them exactly, step by step, and I always catch these sorts of mistakes. Always. I always copy/paste commands because I know that is what the customer will do, so I have to make sure that works flawlessly.
Dude, if you actually follow your own document, you don't have to pay people to do it for you. Also, you can make sure the person you are paying doesn't ignore the document like you apparently do.
I stopped reading, I'm not interested in whatever else this dude has to say. I'm literally flabbergasted.
Maybe it's because my documents have always gone to real paying customers who have to get through this install, and not some hobby project I'm super proud of or whatever and I don't think I'm super clever? Idk.
> I know someone is going to say "why not just ask an LLM to simulate a range of users?" The answer is very simple - I want to speak to real people. People are brilliant! They can make you laugh, you can see their cat when it wanders on to the call, they bring a unique perspective to the problem, and they're really happy when you give them a €25 voucher. Some will gladly do it for free and make you happy!
So what the author actually paid for was to interact with humans and the README checking was secondary - because, really, my own first thought was literally to ask an LLM check and try to follow the instructions in the README and pretty much any decent LLM (including several local ones) would be able to check if they're adequate and even suggest improvements (just don't let them write it for you :-P).
Agents and people catch different things. My agent reviewers find real bugs, like pinch-zoom breaking while a finger is on the joystick. But none of them said "this is boring". I did, on my phone, when I walked my character up to a house and nothing happened.
I know, I know: It's complicated. But have you heard of AI agents?
But I just onboarded 4 interns on a project where all they had to do was
1. Install the Nix package manager
2. Install direnv, enter the project repo, and `direnv allow`
3. Toolchain, git hooks, MCP servers, in-repo issue tracker, everything is available
Our project manager requested information that was available in the issue tracker. I told her, she could get all her answers by asking our agent, and it'd automatically reference the issue tracker. I figured I'd just need to show her how to install the Nix package manager. But no, she already had it because another project by another team depended on it.
Putting wrong information is README is so outdated when you have programmatic setup of your entire toolchain.
I sometimes write readmes for myself, so that I can remember the exact steps to generate a data report, etc.
It's surprising how much they become incomprehensible after just a couple of weeks; when everything's in our head it's all clear, fluid and self-explanatory; but once we have forgotten the context, nothing makes sense anymore.
In school this lead to a lot of difficulties for me, one in writing those comments out for other people to understand, but the other seemed to be that when reading the average statements used in education for teaching I could map the same strings of language to multiple and sometimes conflicting statements because of the inexactness of the language used.
It turns out explaining concepts while leaving little room for different interpretation is hard.
Half the posts I see lately are like, "Gleam 2.0. What we learned" and then you go to the homepage and it's "Gleam is a Tribble for your Fork! (Scroll down) See if you qualify for Gleam Enterprise!"
“You no longer have to florp, now you can vorp!”
Big numbers, random charts. 10x 100x 200x!
Who is this for? Just make the docs the home page.
[0] https://gleam.run/
(Updated the text as the downvotes indicated people misunderstood what I wrote. I agree about OP's observation)
The Nielson Norman Group has a good introduction to this: https://www.nngroup.com/articles/usability-testing-101/
Is this interesting to people on HN?
I majored in a mix between coding and design.
https://www.nngroup.com/articles/why-you-only-need-to-test-w...
https://www.nngroup.com/articles/how-many-test-users/
You want to hire your target audience, which may be vastly different from yourself, and suddenly you discover that the language is throwing them off, the color scheme brings different meanings, and they just don't understand the flow which felt completely natural for you.
I oftentimes find myself spending more time rewriting readmes than writing code.
Treat the readme like a journey/walkthrough of your product, follow an order, and keep it simple to understand.
I used to write technical documents in prose style, sometimes with meandering stories. I guess I picked it up from my early blogging days. I realized I hated reading some of them back. So I tried to keep it cut and dried. I do sometimes sprinkle a bit of colorful wording just to add a bit of humanity but only if it doesn't get in the way of the main message.
[1] https://en.wikipedia.org/wiki/Literate_programming
[2] https://github.com/spicyjpeg/ps1-bare-metal
Well, I found that sentence in the post was very funny ;-)
I tend to be too verbose in writing as I want to explain the context in more detail, but short, accurate, concise is best. Many developers fail at that too, though. Many projects do not have working examples. That annoys me the most. It sends a message of "I don't care about new users learning how to use my project".
I published something the other day with minimal instructions, and felt briefly conflicted.
But I figured, if you want to run it, you'll find a way! (It probably doesn't even work on other operating systems, but porting it would take what, 20 seconds of Codexing?) It was true before AI, and it's definitely true now.
My intended audience is people who want to get their hands dirty. Though I suppose these days, that's the machine's job...
The golang docs are like this. As a novice, you are looking for detailed prose, but as you progress, you come to appreciate the terseness.
It’s normal to compensate them for their time.
Normally though you don’t modify it after each participant. But for something very niche like following a README (as opposed to an e-commerce flow targeted to the general population) it might be fine, if less rigorous.
As I already mentioned in another comment, I recommend this post for people who want to go a bit deeper: https://www.nngroup.com/articles/usability-testing-101/
Even if it is a command-line tool, a screenshot helps provide a better understanding of what to expect.
It seems to be taken for granted — that's something you'd already have in place if you're already serving applications in PHP, but would have to figure out on your own if you haven't served anything in PHP so far in your life.
As I say in the linked article, it depends on what sort of user you have. For a "getting started with Raspberry Pi" document, you might well want to include how to insert an SD card etc.
I'll have a think about the best way to help people figure out if they're running PHP. Thanks for the feedback!
from experience in ENT support where i was sending instructions & quick fix scripts to technically capable persons, you will learn very fast when you’ve missed the mark with your documentation / instructions. tons of times i had ready made solutions that i thought “just copy and paste and go what could possibly go wrong?” and was caught off guard how often a little too much knowledge lends to confusion. i am not blaming the users here it is my fault that i didn’t explain things like “no don’t change this date in the fix that is a special date when the issue could have earliest occurred and it’s there to avoid grabbing more than we need to parse”, but i didn’t tell that so of course people changed it to all sorts of dates thinking they had to
such feedback and issues also got me way better about writing code that avoided chances for such mistakes as i didn’t want users to have to read a novel to understand what to do; it’s a fine balance between what to solve with documentation and what to solve with code
My rule of thumb for my READMEs: there should be a list of commands, that when executed in order and in a clean machine, result in the software doing something useful. Yes, this includes `git clone`.
If there's something the user might already have, like the webserver, I add a comment "skip this if you already have a web server". If there are any shortcuts that make it not production-ready, it's time to break out the ALL CAPS.
Limiting the operations to simple commands also helps me keep honest about the instructions (no hidden assumptions), and forces the software to be minimally testable.
'If you wish to make an apple pie from scratch, you must first invent the universe.'
Like "This manual assumes that you have a Linux/BSD, a C compiler, GNU Make, and a text editor".
Update: Found some related HN threads (omitted link-rotted submissions).
- I'd like to review your README https://news.ycombinator.com/item?id=26842191 (91 comments)
- Readme.so – Easiest Way to Create a Readme https://news.ycombinator.com/item?id=27006740 (65 comments)
- Readme Driven Development https://news.ycombinator.com/item?id=1627246 (57 comments)
https://github.com/matiassingers/awesome-readme
https://github.com/yowainwright/awesome-readmes
I've great success with friction logs: https://mikebifulco.com/posts/how-stripe-uses-friction-logs
If you are a platform team, going through this with your internal customers is both driving adoption and making your tools better. Highly recommended!
I think good technical writing requires the same skills as good product ownership, that is empathy for the user and their perspective. Often technical writing is an after thought and not someone’s whole role and it really shows.
Good article
Also, it features an FAQ. FAQs are problematic (as in not often effective): https://passo.uno/what-the-faq/
Edit: Clarification
Nice, good article lol. I appreciate a joke or two in a readme but totally understand the annoyance, I think forgetting to actually say what the software does is super common as well though. Often trying to figure out if something found on github will actually solve a problem only to be met with a list of install commands.
Paying people to record themselves using your software without reading the instructions might actually be very informative.
I imagine the ideal is a product that requires no manual. That's not always doable, but sometimes it is. Sometimes you can get very close!
You didn't have to murder me like that!
Confession time.
I used to be prone to giving things slightly silly names, particularly unrecoverable structured exceptions. Examples include PancakeLandingException, ReallyBadException, CataclysmicException, ApocalypticDeathException, and the like.
And then years and years ago I used to work for a company called Redgate and I started a tradition of slightly silly messages when early access builds of our .NET products would expire, all based on Monty Python sketches and quotes. So, obviously, the dead parrot sketch featured in there.
So far, so harmless, but this did come to a head somewhat spectacularly and in a couple of different ways.
Firstly, in 2009 one of my colleagues used a modified quote from The Life of Brian as an expiry message on a build. Somebody who appeared to be some sort of religious zealot complained loudly to the company. We were both in LA at the time, with a couple of other colleagues, attending build 2009, so we woke up to a chain of something like 50 panicked emails in our inboxes with people expressing differing levels of outrage and/or amusement whilst discussing various grovelling apology options... until someone figured out that it was actually an elaborate troll that we'd swallowed hook, line, and sinker. I can't remember the name of the person who caught us out but, hats off, well played, sir, well played. It did unfortunately mean we became a bit more cautious and business-like with our early access build expiry messages.
Secondly, and this one needs a bit of context setting... I've always been a fan of descriptive and explicit error messages: there should be enough information in any error message that most of the time the user can figure out what's wrong and fix their own problem OR at least so that if they get in touch with support, then support can quickly figure out the problem and get back to them with a solution. I'm not a fan of unhelpful, information poor, obfuscatory, or cryptic error messages.
But when I wanted to cause an application to exit because there'd been an error related to tampering with our licensing code I played somewhat against type. I wanted error messages that would uniquely identify what had happened, making it easy for us to figure out, whilst giving the user no clue (because I wanted to make it very slightly harder for hackers/crackers - but let's be real: this would never have actually stopped anyone). So I used successive lines of dialogue from a scene in House where House is trying to guess who Wilson's girlfriend is. I have no clear recollection of why I chose this dialogue to reproduce, but... I did.
Anyway, this did lead to some slightly confused support requests coming in from users mostly trying to use the tools legitimately in slightly unusual scenarios, but nothing that was overly burdensome. That was until early 2011, when Greg Young - he of event sourcing fame - posted the following gist because he'd encountered an error that said, "Because I wanna ask you about your girlfriend. I must know who she is, or you would've told me her name.": https://gist.github.com/gregoryyoung/871736.
Not at all creepy, right? And, of course, it went viral on twitter. Cue another massive email thread although, this time round, people just thought it was funny. However, we did decide to make the error messages a bit more boring and, in the end, I just gave them numbers.
Mostly I'm just glad the error Greg got wasn't the final line of dialogue in the exchange between House and Wilson: "Yo mamma." That would have been bad.
It feels like like the AI agents can't help themselves sometimes, and the judgement exercised around what's included and omitted is baffling.
But maybe READMEs have always been this bad, and AI agents have raised the baseline?
As someone on the spectrum, this does happen to me with people sometimes too; I struggle when asking a question and getting an answer that doesn't fit the "shape" of what I expect (e.g. asking a yes or no question and getting a relatively long sentence in response that doesn't contain either "yes" or "no" in it, which means I need to do the equivalent of applying it as a diff to my mental model and seeing if there are conflicts). This happens less frequently with other humans though, and on average the amount of effort I need try to figure out what they're saying is a lot lower. This doesn't make it less frustrating when I get that kind of output from an LLM, but it doesn't surprise me all that much that it happens.
I'm sure I do this all the time to people too, though. One of the biggest lessons I've learned in the past several years is that I communicate in ways that I'd probably have trouble understanding in reverse a lot more frequently than I realized, even I still do the things I find confusing from others less often than most people I interact with. I'm sure a lot of people might find this comment to be pretty much exactly like what I'm complaining about even though I feel fairly confident at least in this moment that my point is clear.
I've have people in my life where I've got situation going to the point I got sick of it and just started responding 'That didn't answer my question. Do you remember what my question was?'. Turns out many people seem to invent out of whole cloth what your question is rather than actually responding to it. Seems like a whole lot of wasted effort. I mean, it's one thing to ask a yes or no question and getting 'it depends on X, Y, Z, blah blah blah', or just 'I'm not sure/don't know'. There have been far to many cases where I ask something and the response is functionally 'well you see, it all began in 1969, back when dinosaurs ruled the earth'.
LLMs seem to atleast actually respond to your question. The question might be phrased in a way which doesn't match what you intended, and the answer will be verbose and require you to wade through prose to actually find it, or it might just be wrong, but it will at least be an answer to the question you gave it.
It was nice when emoji were used sparingly and with purpose and intention.
As my childhood English teacher said - too much of anything, good for nothing.
I kept them because they make me smile.
But isn't the whole point to make your users smile? No one but you cares if YOU smile.
I just find it a bit misleading that you’re saying you want to talk to real people all while trying to clean up AI slop.
None of the code or documentation was written by or assisted by AI.
I will say, if I did stumble on your project, just seeing that file there would be enough to make me skip past and not spend time reading. I look for those before investing any time reading a readme.
It is under 5 minutes to write a shell/bat/make/cmake script for each platform to configure library requirements, import OS specific data, and enable GPU/NPU features. Then run through the application build, installation package with stripped performance build, and or a few regression tests.
Lets say you have 4k downloads a month on a small project, and it takes 1 hour for each admin to read/configure. You just saved about 5 years of your users lives reading your document. =3
Having said that, I found consistently that when a project has working examples, ideally documented a bit, aka explained, they tend to work much better than those projects that have no examples. Working examples often also help get into a project quickly and check out how it works. It helps to learn too.
READMEs are not useless, of course, but the quality varies a lot. I also know of folks who use AI slop spam to improve it, but while it may improve a little bit, it generates a lot of horribly to read text that makes no sense. I am noticing this with the ruby core dev team - they (almost) all suddenly have perfect language skills but it is more like an advanced babelfish translator. What they piece together here makes no sense. Claude in particular is now famous for this slop content. And I don't understand what it is used: real people read any of this AI slop? Because I just skip it or filter it away these days.
Because theoretically you should be able to describe not only your entire application logic, but upper and lower bounds of inputs as well. Good code would describe this inherently.
Documentation is only useful when a) the application is not source available, so you have no choice, b) you want to save a human developer time for them to understand your code, or c) you want to use documentation as a cache hit for agent use (less token spend)
For God's sake, this is not "biases," wtf?? That's straight up just not reading/following the document you're supposed to be testing/reviewing. When I test my documents I actually follow them exactly, step by step, and I always catch these sorts of mistakes. Always. I always copy/paste commands because I know that is what the customer will do, so I have to make sure that works flawlessly.
Dude, if you actually follow your own document, you don't have to pay people to do it for you. Also, you can make sure the person you are paying doesn't ignore the document like you apparently do.
I stopped reading, I'm not interested in whatever else this dude has to say. I'm literally flabbergasted.
Maybe it's because my documents have always gone to real paying customers who have to get through this install, and not some hobby project I'm super proud of or whatever and I don't think I'm super clever? Idk.
So what the author actually paid for was to interact with humans and the README checking was secondary - because, really, my own first thought was literally to ask an LLM check and try to follow the instructions in the README and pretty much any decent LLM (including several local ones) would be able to check if they're adequate and even suggest improvements (just don't let them write it for you :-P).
I know, I know: It's complicated. But have you heard of AI agents?
But I just onboarded 4 interns on a project where all they had to do was
Our project manager requested information that was available in the issue tracker. I told her, she could get all her answers by asking our agent, and it'd automatically reference the issue tracker. I figured I'd just need to show her how to install the Nix package manager. But no, she already had it because another project by another team depended on it.Putting wrong information is README is so outdated when you have programmatic setup of your entire toolchain.