I’ve worked in many different industries writing software, mostly for internal applications but a few external commercial systems as well. Most of my projects have been of mixed success, but I’ve been very good at killing off obviously dead projects before they cost tons of money and wasted effort. “Paranoid” is probably the best word to describe my approach to risk analysis for projects. At the first hint of trouble I start looking for failure causes, and if I can’t get rid of them I either kill the project myself or push heavily for its destruction.
NOTE: This is a DRAFT of the essay. Feedback is more than welcome.
Recently I started doing consulting for a few companies and noticed that many small firms have no clue about when a project will become a disaster. In my previous work the organization was large enough to absorb failures, so I could afford to take a project on and then monitor it for potential failure. With a small consulting firm one disaster project can sink the whole company. Taking on many random projects to see which ones have buoyancy just isn’t an option.
After talking with some of my friends all working for small companies I worked out a simple method to give you a quick idea whether a project is even worth attempting. This method is based on a simple revelation about how clients and projects can be divided generally into Customer,Collaborator;Implementation,Invention. I call this “The C2I2 Hypothesis” since it’s not a theory yet, never will be a theory, is based mostly on my thoughts on the subject, and you should probably just use it as a way to think about the problem yourself.
The core of the idea is very simply laid out into five statements:
- Clients can be either Customers or Collaborators.
- Projects can be either Implementations or Inventions.
- Customers expect Implementations to be finished products and hate
watching you “make sausage”.
- Collaborators expect to help build Inventions and want to help you
feed the pork into the grinder.
- The other two combinations
(Customer>Invention;Collaborator>Implementation) end in
Not Boolean Logic or Mushy Thinking
When you read this essay, try to stay in the middle of “everything must be absolutely classified into true and false” and “everything is anything so I can cover my ass”.
This isn’t a boolean logic decision exercise. Clients will be sort of Collaborators, sort of Customers, and it’s your job to figure out where they mostly stand and whether you can move them to what you need for the project to succeed. Consequently, if you can never place clients and projects into their most likely categories then you won’t be able to make a decision about how to deal with them. I know you managers try to setup every decision so you have plausible deniability, but maybe that’s why everyone hates you. Make a decision for once in your life, just make it intelligently.
When you walk into the grocery store to buy meat, I bet you think it comes from happy little “beef trees” that grow in Montana. You probably grab your nicely shrink-wrapped package of bacon, then a few chicken legs, and probably some sirloin without thinking. It never occurs to you that this was flesh carved off the bones of an animal which shit and pissed itself as some guy smashed a hammer into its skull. Nope, you don’t want to know that. This knowledge would violate your perfect little fantasy that you’re not eating something that used to live.
A Customer does not care about how the meat is made, except maybe to buy particular cuts from their butcher in appropriate poundage. In fact, they become kind of violent when they’re forced to understand the details of animal slaughter. If your butcher made you watch a video of cows being slaughtered before you bought your meat you’d just find another butcher. In the end, a Customer just wants to live in that happy little place where sausage, chicken legs, and other dirty details come from thin air.
To put it simply: Customers are only interested in the end product, not the production process.
Collaborators are different from Customers by their need or desire to understand and participate in the production process. Maybe they have money, or they’re insane control freaks, or they have a particular vision and don’t want it screwed up. Either way, a Collaborator will hire someone’s services not to just get a product (like a Customer) but instead to also contribute to the production process.
The other part of this little decision matrix is the Invention vs. Implementation aspect. The simple definition is:
“An Implementation is something you’ve done many times before and doesn’t need as much collaboration from the client. An Invention is something you have limited knowledge of and therefore need an outside collaborator to get right.”
The crux of the difference between Invention and Implementation comes down to knowledge. If it’s something you or someone else has built a million times before then it’s an Implementation. If it’s something you’ve never built before then it’s an Invention. Notice I didn’t say if it’s something that’s actually new, but that it’s something that’s new to you.
Let’s break from the butcher analogy and switch over to a mechanic. This works better since mechanics can be hired to fix your car or (if they’re good) build you a weird new one. In between these two extremes is a Custom Implementation such as what you’d buy from Orange County Choppers.
In the case of the weird new car the mechanic is making you an Invention. He’ll need to collaborate with you probably constantly, always asking for your input, your vision, extra money, and he probably can’t tell you when it’ll be done. You’ll just have to sink money into the process and pray it works as you’ve planned. The reason is you don’t have the knowledge so you have to get your vision made by proxy. This is long, arduous and requires collaboration.
However, if you roll into a mechanic’s place with your messed up Gremlin and try to “help” the mechanic fix the timing belt and plugs then he’ll charge you extra or may throw you out. Why? It costs him more to put up with your lame Wannabe-Collaborator ass than to just do it himself. He has to explain important details about things already common knowledge in the industry. Combine this with the fact that since this is an Implementation of a repair the mechanic does all day long, it’s incredibly boring so dealing with you is making it even more frustrating.
The in-between though is a Custom Implementation. Now, if you’ve ever seen an episode of American Chopper you’ll know that OCC makes neat custom bikes for rich people. You’ll also notice that while they want the customer’s input and collaboration in the beginning, they don’t want the customer’s help in building the bike. In the case of a Custom Implementation, the OCC guys Collaborate first to get the vision, then they Implement for a Customer.
It’s important to know this distinction because I know everyone reading this is screaming “I can just claim everything is a Custom Implementation and have the best of both worlds!” No you can’t. OCC still doesn’t involve customers in a ton of the chopper building process (they do involve viewers though). OCC is also very good at understanding the customer’s vision, and turning that into a creative work of art that has a solid consistent core.
In other words, OCC collaborates on the appearance, but deep down they build the same damn bike over and over again. Sure the chrome is different, and maybe they lay in a new engine but ultimately, it’s still a chopper, baby. Additionally, OCC knows a ton of crap about building choppers and are grand experts at it. They don’t need customer’s help and can do them very quickly once they have the vision right.
Going Wrong In All The Right Ways
We’ve laid the foundation for a simple way to classify clients and the projects they want you to build. A warning about this classification system is that it’s too simple but remember it’s just a guide so that you can get clearer about the projects you are going to take on.
So far we have defined:
- Customer – Someone who expects a finished product and can’t
stand watching you make it.
* Collaborator– Someone who wants or needs to be involved in
- Invention – Something of which you have limited knowledge and
* Implementation– Something you’ve done many times before.
- Customer goes with Implementation; Collaborator goes with
With these definitions in place we can start exploring how projects go wrong due to bad combinations of the types of clients and projects. We can also explore how to move your project or customer into something more beneficial.
Don’t Make Me Slaughter Lambs
Let’s continue our butchers and the meat industry analogy to point out how Customers and Collaborators go wrong.
A butcher will put himself out of business if he insists on including Customers in the sausage grinding process. They don’t care, even if they really pretend they do care by selecting cuts and sizes. The butcher who forces each customer to go down to the slaughter house and help smash sheep skulls, carve carcasses, and rend flesh will not have any business.
Yet, this is what a lot of programmers think customers need. Just take the eXtreme Programming (XP) process as the ultimate example of this. XP forces the customer to observe nearly every aspect of the software design and construction process. They are there constantly dispensing advice and decisions as if the programmers have no ability to decide for themselves. Customers hate this crap more than you know, but Collaborators love it more than their first-born children. Of course when a Customer is forced to do this they’ll put a fake smile on and grind through it, but in my experience they’re already starting to look for ways to kill your project off quick.
Remember it depends on the project whether a client can be a customer or not, and whether this would be a risk. In most enterprise-style software you actually can force a customer to become a collaborator if you need it. Consumer software doesn’t give you this luxury since there aren’t customers wandering around that will freely give you exact specifications for your ideas. So, even if a project is an Invention and the client is a Customer, you can still do some things to turn them into collaborators in the right situations.
A Collaborator can really screw things up if she gets involved in the production of a product that doesn’t need her help. Butchers and the rest of the meat industry don’t need your advice. The second you go running behind the counter to tell them how you want your particular piece of bratwurst ground and cured is the same second you screw up the bratwurst. It’s an industry that’s literally thousands of years old, so collaborating with them is pointless since there’s nothing for you to contribute.
The same applies to software that’s been done many times before. It’s more important for a manager or “champion” in this case to take a more customer role by giving good requirements up front and letting the team do its thing. Collaborating on an Implementation just causes waste and makes the project take much longer than it should. By being a gadfly you are only causing trouble and not really helping, especially if your “collaborating” is nothing more than a thinly veiled attempt to shift project failure blame onto the programmers.
Now, if you are the one financing the operation that doesn’t mean you have to be involved like you are actually a Collaborator. Having the cash just means you get to watch over what’s going on, but really you have no control and trying to exert some kind of control will just sink the project. Back up and instead try to watch from a distance and use metrics to track progress.
Poor Individual Snowflake
This is where most programmers go wrong since they desperately want everything they do to be original and important. For them admitting that they just make web sites that are all the same is admitting defeat. They can’t say they spent 4-6 years of their life learning Scheme and AI at MIT to just write Yet-Another-Web-Site.
What programmers end up doing is taking something that could be an Implementation, and then magically adding on huge amounts of complexity so that it has to be an Invention. Then they invent processes that involve Customers at every stage and force the Customer to devote entire full time employees to the process. Yet, in the end they crank out Yet-Another-Web-Site.
Let’s take the famous project that launched XP. It was an Implementation not an Invention. How do I know this? Well, they said it was a project to replace an existing system. Sounds like something that was done before. That’s an Implementation. No need to change it too much, invent new ways it works, or get ambitious. You’re just making a replacement and changing it too much will add to it’s potential failure.
If that’s the case, why was the customer involved all the time? They had a completely working specification in an already working system. Replacing it is more a matter of reverse engineering than gathering vision, customer feedback, use cases, stories, or any of the other crap the XP team used. In the end, they made the customer watch the team slaughter cows and grind sausage. In the end the company threw all the rotten meat out when the project never finished.
Now, this doesn’t mean I think XP is crap. What I’m pointing out is more that XP is a process that works for building an Invention for a client who needs to be a Collaborator. In the case of the original XP project, they had an Implementation and a Customer, so they should have used a different process with less Customer interaction and more reverse engineering or system specification.
This is the first way that the C2I2 idea can help you avoid a potential failure: Not using the right process for the project at hand will kill it off eventually. You can still make ass-loads of money bilking the client with dreams of being involved in every aspect of their next generation purchasing system that works just like the last one. However, the project will be much more efficient if you bite the bullet and just admit it’s nothing new.
No Clues To Be Found
If “snowflake fantasies” are what happens when a programmer forces an Implementation to be an Invention, then “clueless moron” is what happens when you assume an Invention is an Implementation.
Remember that what makes something an Invention is whether you’ve known about it before and done it already. Read all the books you want, but without this experience you’ll go through a lot of learning and pain and mistakes. In a crunch this can be costly.
What happens is programmers arrogantly proclaims “Oh yeah, that’s easy!” Either way, you know you’ve done this when you get about 1/10th of the way into the project and you realize you have no idea at all what you’re doing.
How do you fix this? Simply hire someone whose done it before. Hiring a domain expert who’s implemented this before is about the only way to avoid the problem. If you can’t do this, then reclassify the project an Invention and make sure you give it lots of money and time.
Or, you could just kill it off and save your money. Silk purses and sow’s ears and all.
This is still an idea I have about projects and clients. Maybe you’ve got a great example of where your project failed for similar reasons, or other advice. If you can figure out my e-mail, then send me your experiences for my follow-up essay.
Getting To Beneficial Positions
You’re screwed. You’ve got a project and a client that are just totally wrong. The project is an Implementation and you’ve got a meddling Wannabe-Collaborator who’s constant questions and horrible decisions are making the software suck and take ten times longer. Or, you’ve got an Invention you know nothing about and the client is a Customer who expects perfection on the first viewing.
Maybe you’ve got a project or client mis-categorized. Programmers are telling you the project is easy but they’ve never done it before. Managers need to be collaborators but are flipping out whenever they view unfinished prototypes early.
My general philosophy is, “Kill it early and save your money.” There’s this huge myth in IT that we’ve got more failures than other industries. The problem is we’ve defined “failure” as “not finishing despite all adversity.” Most other industries don’t do this and will kill a project early if it is obviously not going to make it. We’ll ram a project through no matter what, even if it’s obvious that it’ll fail miserably. In IT, we’ve missed the idea that we can just avoid projects or kill them early and avoid spending all that cash.
Normally I just avoid bad projects to begin with rather than take on potentially damaging work. Life is too damn short to spend it on useless software, and trust me I’ve wasted a lot of my life on some of the most useless software ever.
In a professional setting where someone else is paying me to build them something, I’m in the same situation as everyone else and have to either avoid the project or do things to move the project into a beneficial situation. I’ve got a few ideas and techniques I use to do this, but feel free to shoot me more suggestions.
You can also tell from the diagram above that I have a view on what development processes work the best for the different situations. This is just based on my personal experience doing this for a long time, but I’ll also explain my reasoning.
Implementation to Invention
Moving a project that is assumed to be an Implementation into the Invention category usually takes convincing everyone that it’s more difficult than originally estimated. You know you’re in this situation when you read this description:
“The project will accept any protocol from any client that talks to any mobile device or video system and will involve the following 35 different Web 2.0 technologies using an SOA architecture. The project should take six months.”
I have no idea why every project is always under-estimated at six months. It’s like some weird magic number. I generally assume that any project that has a 6 in its estimate is based on a bogus estimate.
When you see these kinds of projects you know you’re dealing with an Invention since it’s obvious there’s very limited knowledge of the actual problem. Usually these estimates are not backed up with any kind of planning metric, don’t have any reasons why they can be done so quickly, and they’re almost always planned by a desperate manager or an over-confident engineer or both.
I just kill these projects. My experience has been that too many people put too many asses on the six month deadline and then they abuse the programmers for the failure. These projects always overrun costs, are horribly implemented, and are never done on time. If they are “finished” by the six month deadline it’s only after much fighting and firing and lost features.
Be careful with my tactic though, since this quickly makes you many enemies. The worst thing you can do is be right on project estimates without being in charge. I’d say every project I’ve ratted out for being under-estimated has caused the manager to hate my guts. When I’m in charge it’s different.
If you’re totally stuck and have no way of getting the estimate changed and you really can’t get off the project, then your best weapon is an established project estimation metric and a small memo detailing your estimate with a request for the original project plan to be evaluated in a similar fashion.
My personal favorite are Function Points combined with evaluations of past project FP measurements controlled by code size and rate of FP completion. It doesn’t really matter what metric you use, just so long as it’s industry established and you can back it up. The reason why I advocate a metric is they’re harder to game. Basing the metric on past project experience is also a good way to tune it to your organization.
Here’s an example to give you a quick idea. Let’s say I see a project that’s estimated at June but it’s huge. What I do is the following:
- Run through a quick FP using the Tiny Tools FP Calculator to get a quick estimate.
- I then calculate the complexity settings at the simplest and most complex. This gives me a range of possible values that work as a best-case vs. worse-case scenario.
- I then pick a project that the same company has done in the past (or three if you can) and do the same FP calculations for them.
- Using this information, I then look at how long these previous projects took, and how much code was written in SLOC. What I’m looking for are two metrics: FP/day/developer and FP/SLOC.
- Then, I take the FP/SLOC and the FP/day/dev metrics and use this to figure out the speed at which the project could be completed based on an average person’s typing speed if all they did was type it straight into the computer. You’ll see why.
- Finally, I use the FP/SLOC, FP/day/dev, and FP at typing speed and compare with the proposed project’s projected deadline. What you’ll find out many times is that the project is under-estimated so much that it couldn’t be done even at typing speed. I’m not joking.
Now, FP has a really bad rap because idiot managers use it as a measure of programmer productivity. Don’t do this. They are better as a planning tool, and they are only valid when combined with other metrics as controls. Thus the FP/day/dev and FP/SLOC. FP by themselves are totally useless magic numbers, but when you use them as a consistent metric across multiple projects they help you plan out potential projects very quickly to see if they’re even close to realistic.
Once you’re armed with your estimates, you’ll have a better chance of convincing management that things are going downhill.
The Domain Expert
Once you’ve got the project finally classified as an Invention and it’s obvious you don’t know anything about the domain, then hire a domain expert. These don’t have to be technical people, but could be someone with tons of experience in the field you’re trying to target. A developer who’s worked on the same project before or a product manager from a competitor are also great domain experts.
The key is to increase your knowledge and experience about the project. The more information you get the better off you’ll be and the easier the implementation will be later. This means that planning your project with more up-front design is an essential activity.
Another thing you can do is simplify the project down to its bare minimum requirements. The less there is to do the less you need to know about it.
Finally, you’ve gotta use an incremental development process with tons of client feedback and make sure the client knows their role as a Collaborator. If they aren’t then your project will fail for sure.
Invention to Implementation
Implementations usually are classified as Inventions by ambitious and idiotic developers looking to stuff their resumes. Sometimes they’ll be classified this way by control freak clients, but since the client has no real control over the process you can usually just change it to an Implementation without much protest. Convincing a developer that his “blood diamond” is actually just a chunk of asphalt is much harder.
This is probably the 1.1 cause of Architecture Astronautics and it’s pretty damn hard to stop. Developers are notorious for taking simple stuff, declaring the simple solution “inelegant” and making a giant horrendous Rube Goldberg Architecture.
My favorite solution is to fire that guy. I’ve been involved in too many projects with obnoxious software architects who really don’t belong. Just get rid of them and hire someone who not only designs the systems, but also helps build them. Eating your own dog food is a great cure for wanting to build gigantic systems.
If this isn’t possible, then be prepared to piss off every developer on the team since you’ve basically gotta come in and trim the fat yourself. First thing is to challenge every design decision. Demand a full architecture diagram, and then have the developers on the team take it apart and boil it down to the simplest thing possible.
Next thing you do is rip out feature requests unless they are absolutely needed. Practice ruthless “You Aren’t Gonna Need It (YAGNI)” at every turn. Question every giant purchase and ask for three alternatives that are cheaper or simpler.
Finally, if you’re not a manager and you’re watching this drama unfold where the software architect throws massive UML diagrams over the wall for you to suffer through, then your best weapon is your coding skills. Smart clients will pick something that works 80% now over something that will take 2-3 years and might not work.
Collaborator to Customer
You’ve categorized your project appropriately and done all the things to manage that type of Invention or Implementation. Your problem now is that you’ve got this Wannabe-Collaborator who really needs to shut-up and be a Customer. This is similar but different from the problem of an Invention that should be treated as an Implementation. The main difference is that everyone agrees that this is an Implementation, but the problem is the client isn’t interested in participating as a Customer.
Again, the best thing is to educate the client about what their role will have to be. You’ve gotta tell them how they’ll fit into the project, frequently get their input up front on what they want, and then really nail it when you give it to them. It’s very important that you’ve done this kind of work implementing this same thing many times for past clients.
Keep in mind I’m not advocating keeping the client in the dark about the progress or the results. If they want frequent updates and to see work as it progresses then that’s great, but they’ve gotta be hands off and avoid changing direction too violently. They also should never be in control of the development process. If you have a client who’s bringing you articles from CIO magazine about XP then you’ve gotta sit them down and educate them again on their role.
If it’s not possible to educate the client then your next step would be to hire someone to babysit them during the project. You also need to charge more money just like a mechanic would. Watch out for the con-men as well. Many times I’ve heard of these types of people claiming they’re not satisfied with your work at the end. They’ll refuse to pay, but gladly take your work. Get a portion of the money up front if you suspect they might do this.
Customer to Collaborator
You’ve got an Invention but the client is acting like a whiny Customer. Every time you show them prototypes, semi-functional systems, or project plans they freak out. “This is what it will look like?! That looks like shit! Why isn’t it done already?!”
People have a really distorted view of how long software takes and what the process is for making it. They’re used to shrink-wrapped products they buy from companies that are fully formed and work without much intervention. Getting them used to the process will be very difficult and is gonna slow you down tremendously.
Most of the folks I talk to in this situation simply say, “Only show them final versions.” This works really well, but it’ll take you longer since your project is an Invention remember? You’ll have to assume that you’ve gotta show them many “final version” prototypes in order to get it right, and it’s gonna take a long time. You also run the risk of getting it wrong and having the client get pissed.
I prefer to educate the client about how the project needs their constant input, schedule regular meetings they need to attend, and don’t use a fucking computer. That’s right, don’t show them a single computer screen during your meetings. Show them paper of all prototypes and make it incredibly clear that the real program will look nothing like this. Say it over and over and over until they get bored with it and start asking when they can see the real thing. In fact, use giant sheets of paper with crayons. Anything to hammer home the fact that this isn’t the real thing, so they shouldn’t freak out about it not being done right.
You also have to avoid asking them things over and over again. Take extensive notes, collect the notes, categorize the notes, translate these into feature requests and plans, and only ask to clarify things that are unclear. A client who acts like a Customer on an Invention project hates being asked the same crap over and over. It makes you look incompetent and frustrates them endlessly.
Otherwise, I’d say Customer+Invention is probably one of the most difficult project situations but if you can manage it well then it is very rewarding. My personal opinion is again just don’t take these projects on. Skip them unless you are able to charge a huge fee. You really want to have Customers+Implementations as the project business.
Appropriate Development Processes
This entire essay is all based on my opinion and experience managing these wildly different projects with a variety of successes and failures. I’ve used many different development methods and studied them in graduate school. If there’s a section that will be controversial it’s this one.
I’ve found that only three development processes are really worth trying out these days: eXtreme Programming (XP), Scrum, and Capability Maturity Models (CMM). Yes, I can hear the blood boiling and the fingers flying at the keyboards as people rant against me about their favorite development process. I’m not saying your process is bad. I’m saying I prefer these three depending on the situation, and I’ll even mix features from them to adapt for specific projects.
In reality, I don’t use XP or CMM but end up using Scrum most of the time because nobody has perfect little projects that fit neatly into Collaborator+Invention or Customer+Implementation sweet spots.
If you look at the diagram I did again:
You’ll see that I prefer XP for Collaborator+Invention projects, and CMM for Customer+Implementation projects. These two situations are fantasies though since most real projects tend to force themselves into the disadvantaged situations (Customer+Invention or Collaborator+Implementation). If you can move the projects into the sweet spots you prefer then you’re set, but many times people try to force the project and client into the role that fits their process, rather than picking a process that fits the situation.
XP is great for Collaborators working on major Inventions because it has tons of repeatable client feedback, advocates being able to wildly change and refactor the code at a moment’s notice, and frequent feedback on the progress of the project. This is perfect since nobody really knows what the Invention is until they build it, and Collaborators don’t mind being involved in the slaughter house.
CMM is great for Implementations you do for Customers because it advocates heavy up-front planning, repeatability, measurable quality for projects, and lots of tracking. If you’ve done the project a billion times before then you aren’t looking for the ability to wildly change the project, but instead you need to gather customer requirements and make sure that your projects are all efficient. Many times organizations that make Implementations for a living are also skating on the edge of profit margins since these markets are full of competitors. With CMM they can start doing effective Quality Control on their process to make themselves more efficient than their competitors and compete on cost instead.
That leaves the “you’re screwed” positions that probably make up the majority of projects out there in the real world. I prefer Scrum for either of these situations because it strikes a good balance between insane planning found in CMM and fast agile iterations found in XP. With Scrum there’s measurement and planning, but it’s flexible for different situations and the change/implementation cycle is a bit longer (one month) so there’s time to learn about the project and become experts.
However, the real reason I prefer Scrum is it’s a great methodology for projects that you might have to kill off early due to risk. Because Scrum has a simple but easy tracking method for project progress and these measurements are done incrementally you get a very good idea of whether the project is gonna be completed on time. If the client is making huge changes at every Scrum, or if you don’t complete nearly as many features are planned during each Scrum, then you can kill the project rather than lose more money.
Another important feature of Scrum in these situations is its postmortem meetings. The other methodologies have them, but Scrum makes a point of getting actionable feedback from both the client and the development team. This is another checkpoint for the project that you can use to kill it if everyone seems completely unsatisfied with the progress.
I’ve found with XP that the practitioners tend to be zealots and completely ignore any feedback which says the project is failing. In fact, I’ve never been on or heard of an XP project that the XP team has requested be killed. It’s always been management killing the project after massive cost overruns. Of course that’s just my opinion so your own experiences may be completely different.
This essay is mostly a large brain dump of my thoughts on the problem of analyzing projects for potential success. Hopefully being able to use a simple mental framework of Clients, Collaborators, Implementations, and Inventions to get a grip on your project complexity will help you avoid failure in the first place. At a minimum it should help you recognize when what you’re doing isn’t working and give you a few pointers on improving your situation.
I’ve gotta warn you though that you shouldn’t take this as truth or gospel. This is again just my thoughts and experiences. I’ve got little if no evidence that my ideas are valid since there simply isn’t any. Nobody else has much evidence of projects, let alone a decent classification system or definition of different types of projects and clients. Take the ideas in this essay, think about them and feel free to debate the concept and come up with counter-examples or even develop experiments and studies if you can.
The last thing I want is for people to start running around with printed copies of this essay demanding that everyone follow it like Mao’s doctrine.
Thanks to Olle Jonsson for shooting me a nice set of grammar edits.