Fake personalization in tech isn't fooling anyone
If your tech is pretending to be something it's not, it won't work in the long run
Most people understand why material dishonesty is bad.
True hardwood flooring is more beautiful to us than laminate. We like a solid oak tabletop more than a cheap piece of particleboard from IKEA.
They look similar from a distance. What is the input resolution that would be required for an image classifier to distinguish between hardwood and laminate? How many people with PhD’s would you need to answer this question?
Yet the average person can easily tell the difference just by walking on it. It’s in the grain, the way our body heat transfers, the way it bends and creaks. The use of true hardwood in a high quality home is more than just status signaling. It’s an objectively richer experience.
We naturally turn up our noses at materially dishonest things. They strike us as cheap and shameful. Yet tech is constantly pretending to be something it’s not, and we tolerate it. In some circles it’s actively lauded as an engineering marvel or an act of sophistication.
This is wrong. I will present two examples to illustrate why.
1. “Personalized” generic content is an attempt to get something for nothing
Everyone has seen an embarassing mail merge failure: “Dear [FIRST_NAME] [LAST_NAME]”.
But I recently encountered a more subtle version of this on a popular marketing blog:
I use Chrome extensions to block scripts and 3rd party cookies, and I never allow a website to have access to my location. So I got their fallback placeholder: “Your City”.
Natural Language Processing enthusiasts out will note that this fallback value has created syntactic ambiguity. It’s no longer clear whether “will it be yours” is referring to my city or my business. It’s also weird that Your City is capitalized, because normal people don’t write like that.
The jarring, hacked-together nature of this sentence has now distracted me so much that I have to open Chrome inspector to confirm my suspicions. Lo and behold, there is a
<span> with an ID tag wrapped around the words “Your City”, confirming that this field is meant to be programmatically populated:
Materially dishonest tech, example 1: making an interaction seem local when it is not.
We like local businesses for a reason. They are embedded in our local networks, which provide ways to get information about them and keep them accountable.
Would your rather buy a loaf of bread from a local baker or on Amazon? Most people still choose the baker. If the bread is moldy and stale, you can use the full arsenal of human social tools (an angry look, bad word of mouth to your neighbours) to create consequences for their negligence. If the bread is delicious, you can head back to the shop tomorrow, heap lavish praise on them, tip extra, etc etc.
This fake localized CTA was an attempt to gain all the trust and benefits that come from being a local business, without the accountability of actually being one.
This is an attempt to get something for nothing. And humans don’t take kindly to that.
(For fantastic related blog post, check out Signal’s excellent but short-lived Instagram ads.)
2. If it’s not a real person, don’t try to make it look like one
Most of us feel sheepish when we bump into a pole and then apologize. We usually follow up with “oops”, because we don’t want others to think that we talk to objects. People who do this are considered crazy.
So why would you put a machine in front of someone and try to trick them into thinking it’s human?
Yet companies do this all the time.
It has become trendy to send mass emails with real first and last names in the name field, to bait someone into clicking it:
But the gains with this approach are short-term. If your content is mediocre, the annoyance at being duped might be enough to earn you an “unsubscribe” or “mark as spam”. If your content is good enough to keep them subscribed, then this trickery wasn’t necessary in the first place.
Materially dishonest tech, example 2: Making an interaction look human when it is not
I recently encountered a particularly noteworthy example of this - noteworthy because of how much work was put into the hiding the true nature of what was going on. Details fuzzed for privacy:
A few weeks later:
And here are the email headers:
A real email was entered into the “name” field of an admin email address, to make it seem like they wrote it;
The templates for the initial vs the followup vary in writing style (polite business formal vs a quick one-liner), just as a person may do in real life.
I have to admit, this is almost admirable in terms of effort. If most automated mail-merges are laminate flooring, then this is laminate flooring with wood-grain texture stamped in, dipped in a faux-oak scent.
But it is laminate nonetheless. As one might expect, the “connection” in this case never materialized. Given that the value of an introduction is mostly in the relationship of the people doing the introducing, it stands to reason that a “connection” created with zero human involvement will hold exactly zero human value, no matter how hard it tries to put out appearances.
There is nothing wrong with automation per se. Everyone likes ATMs. I get my money faster, and the bank teller is spared the monotony of dishing out $20 bills, leaving them free to deal with more interesting requests.
But humans are expensive, and material dishonesty in tech is usually an attempt to capture human upside without doing human work. This is impossible.
Here are passages from The Seven Lamps of Architecture, by John Ruskin. Chapter 2 is about Truth:
Surface Deceits. These may be generally defined as inducing the supposition of some form or material which does not actually exist; as commonly in the painting of wood to represent marble, or in the painting of ornaments in deceptive relief. But we must be careful to observe, that the evil of them consists always in definitely attempted deception [...]
[...]To cover brick with plaster, and this plaster with fresco, is, there fore, perfectly legitimate [...] But to cover brick with cement, and to divide this cement with joints that it may look like stone, is to tell a falsehood; and is just as contemptible a procedure as the other is noble.
Nobody minds cheap laminate flooring in a child’s playroom: it is installed with the understanding that it is meant to be covered with crayons, dented, destroyed, and replaced in a few years’ time. There is nothing dishonest about it.
But we all know the disappointment of, say, paying top dollar for a charming cabin with beautiful photos on Airbnb, only to show up and discover that the “fireplace” is an LED screen, the plants are plastic, and the beautiful “wooden” breakfast nook is really made of particleboard. In such situations we feel deceived. If the customer service from the host turns out to be poor, we may be further disappointed, but we won’t be surprised.
If the environment around us is fake, it is likely that the person who created that environment will be false too. We will learn not to trust them.
This is as true in woodworking as it is in writing code. I’ll close with this appeal from Ruskin:
"I would have the Spirit or Lamp of Truth clear in the hearts of our artists and handicraftsmen...it is, indeed, marvellous to see what power and universality there are in this single principle, and how in the consulting or forgetting of it lies half the dignity or decline of every art and act of man."
Permanent link for this post: https://casey.li/fake-tech-material-dishonesty/