Writing

Feed Software, technology, sysadmin war stories, and more.

Friday, December 28, 2018

Handling intent for Grandma World

Intent is an interesting thing. It comes up in places where you're trying to "do the right thing" for someone else. It's like the situation where you can see someone trying and failing to send you an e-mail because you're watching the logs and you can see the "user unknown" bounces going out. You can just pick up the phone and call them. Or you can mail them first. Whatever.

The point is, we, as people, are not just collections of rules which run ceaselessly and without variation on solid, reliable hardware. Quite the contrary. It's what makes us flexible when things call for it.

So then, imagine the case where you can quite clearly see someone trying to accomplish something, and then you deliberately ignore it, or actually continue screwing it up for your own benefit. You know exactly what's going on, and what they're trying to do, but you continue to thwart it, because it's better for you that way.

That's evil, right?

Here's a scenario I heard from a friend. Let's say someone is using a recipe-sharing site where they can upload recipes, post videos of them actually cooking and baking stuff, and so on. They can also interact with the recipes and videos and other media from other folks who do the same thing. Call it Grandma World or something.

Anyway, occasionally, one of the users decides they'd rather not have their stuff posted on the site any more. Maybe it's become more trouble than it's worth. Maybe it's costing them money. Who knows. The point is, it's their content, and they want to remove their footprint from the public eye.

So, they load up a web page that lists all of the interactions they've had with the Grandma World site. This "GW activity log" shows everything they've done, complete with a link to the actual item so they can click on it and see it in context. Also, once you've clicked through to the original item, you have the ability to edit or even delete whatever it was you did.

In other words, a user could slowly go through that list, click on each item, let it load, and then click delete for each one. It would take a very long time and would be very annoying and fiddly, but it would probably accomplish what they wanted.

Or, you know, they could have a small script running in their browser which would do this for them. It's easy enough. First you load up the activity page, then you turn this thing loose. It loads up the context pages and then triggers the delete method, which fires off a request at the GW web site, and then it goes away.

Only... it didn't. I mean, sure, sometimes it would kind-of work, but mostly it would fail. It has to do with the fact that a script can run far faster than a real human can, and so it can generate tons of requests per second. It can generate them faster than the GW backend can handle it.

Grandma World was never designed for this kind of mass-deletion on its storage systems, so it would choke after the first few. This was obviously noticed by someone, and passed along to those who supposedly owned the problem.

Did they fix the problem to make deletes able to run at high speed? No.

Did they fix the problem by making it queue up the deletes and then run them at whatever rate it was able to do? No.

No, they ignored it. It was better to not let people bulk-delete their stuff from Grandma World because then maybe they wouldn't leave the site completely.

What team would have been responsible for handling this endpoint? Strangely enough, they were probably called 'growth'. Would it make sense for them to make it easy for people to leave? Probably not.

"It is difficult to get a man to understand something, when his salary depends upon his not understanding it"

-- Upton Sinclair

It all makes sense now.

By the way, you know you're onto something evil when the response to asking a question is to call an in-person meeting and "take it offline", that is, away from recorded e-mails, chat logs, bug updates, and code patches. It's a whole lot harder to bring that kind of event onto the record.

Maybe that's why it happens so much.