LinkArchiver stopped working this week when Twitter turned off the User Stream API that it relied on. Under the hood, LinkArchiver was only looking at its timeline, so it could use Twitter’s built-in following features to make its user list. Since that API change, it can’t pull down a “stream” of its timeline, and so would have to be redesigned to continue to work.
Even as this project is shutting down, I consider it a pretty major success. I am very grateful to Jacob Hoffman-Andrews for pitching me the underlying idea. Writing the code (and seeing it get an enthusiastic reception) was a great way to kick off my time at Recurse Center last summer. I’m also grateful to Ben Cotton who gave it a nice write-up at opensource.com when it launched.
I’ve had a few people ask me about archiving and backup options now that this is no longer available. I’m considering doing something similar for Mastodon, or for plain RSS feeds, but I also don’t want to downplay the fact that Internet Archive does a very good job of running the Wayback Machine crawler, and so the main value I can add is adding a personal layer. In any future work on things like LinkArchiver, I’d want to keep track of that.
There’s also a way to do a Twitter redesign with existing APIs, probably. Instead of getting a stream from Twitter that pinged on new tweet events, it could request new tweets at regular intervals, using an API that’s still operational. If somebody wants to write that, they’re welcome to, but given the way Twitter is, I’m not eager to do so.
“Quote tweets” are treated like links to tweets, and constituted about a third of the total links. Something like 4.8 million links backed up were at domains other than Twitter. ↩
Twitter should allow users to “hide” old tweets so that they are only visible to that user, and selectively “un-hide” individual tweets from that collection so that they once again become available at their original URL, in quote tweets and threads, and in sites where they are embedded around the web.
The need for a mechanism to remove old tweets is very clear and becoming abundantly moreso with every rightwing campaign trawling for old statements to weaponize by taking out of context. Numerous guides have been written about how and why to delete your tweets, and it’s beyond the scope of the argument I’m making today. Many of these guides, too, address the emotional difficulty of actually pulling the trigger and wiping an archive—in many cases spanning a decade or so of interaction.
One of the best encapsulations of this problem (and that solution) is John Herrman’s Awl 2015 piece “Time Is a Privacy Setting.” Even just the phrase in the headline is an important contribution, and gets at why simple dichotomies of “public” and “private” are so unsatisfying. A lot more goes into our understanding of privacy than a binary toggle, and time is certainly a factor.
And just as interesting, to me, is something Herrman said this week, in a tweet thread that began with his a link to that now three-year-old essay: “You thought you were talking, but you were writing.” It’s true! Twitter has managed, for many people, to clear out a barrier that can otherwise be insurmountable, letting people be comfortable writing down their thoughts and conversations. Something the tweet-deleters flirt with, but never seem to quite put their finger on, is that writing (as opposed to talking) has some real advantages in addition to the drawbacks.1
This observation is obviously not new to Twitter. My father, a writer, has always exhorted me to keep a journal or a diary, though I’ve never really been able to. One real value of Twitter, for me, has been breaking through that hesitation by setting the threshold of participation so incredibly low that I can just, y’know, write. It’d be better to keep a journal or to blog, but in many cases tweets are good enough.
Over the decade that I’ve been on Twitter, I’ve gone through periods of prolific writing and total dry spells. I’ve maintained this blog regularly at times, and at others it’s sat dormant for months. And the thing is, I frequently refer to and read the old entries for time periods where they exist! Human memory is fickle and fragile, and it is genuinely interesting and educational for me to look back on old writing and see the germs of ideas that later grew into real passions. Looking up old tweets isn’t quite as good, but it fills in the gaps.
Beyond just lowering the threshold to producing such writing, Twitter did some important things to make online writing even more valuable. Search, the villain in recent sagas, is a powerful tool when used for good. Joanne McNeil recently raised a use case I love:
One of my favorite things things to do on twitter is search my friends’ old tweets for their reactions over the years to whatever film or book I just finished
I do that too! Yes, it’d be great if all my friends blogged about every movie they watched or book they read (and I had a good way of finding those posts when they became relevant to me), but this is a decent runner-up. If you want, too, you can see first interactions with people who later became important to you, or all the people who’ve made a joke you just thought of, or people discussing an article you’ve just read, and Twitter (helpfully, in this case) collapses the time gap between you and those people.
Beyond that, Twitter puts a time stamp and a URL on each of these things. In recent years it even preserves the semantics of threads. I don’t want to downplay the very real harm these attributes can cause, but it’s also true that in some situations they provide a ton of value.
Which brings us back to tweet deletion and why it is emotionally difficult for people to do. For better or for worse, this archive we’ve created of our own writing is valuable, and deleting tweets is permanent. (Emily Dreyfuss’ article at Wired does an especially good job at grappling with it.) Every guide I’ve read to deleting tweets walks through the process of creating an “archive,” but even doing so misses some of these benefits of keeping old tweets online.
Archiving-then-deleting may preserve a copy for you, but it permanently rips each tweet out of its context in threads, at a URL, within quote tweets, embedded in articles, and in the Search index.2 Maybe you want to do all those things, and you should have the power to do so! But with a corpus of tens of thousands of tweets, in many cases, it is overwhelming to permanently make that decision for everything all at once.
(Side note: that permanence is in large part a result of Twitter’s centralization. Under a more decentralized model,3 where we controlled our own domains, you could maybe just rebuild your database from a backup and point all the right URLs to all the right content. There’s pros and cons to such centralization—and in particular, people seem to love the guarantee that the archive hasn’t been tampered with—but in any case that’s the situation we’re in and what can you do.)
So here’s the solution, referred to up top. Twitter should give users a new option to “hide” old tweets, which would look to every other user like deletion, but which would be a reversible operation. Your hidden tweets should still appear in your searches and views of old threads, but greyed out to indicate they’re not otherwise available. And—and this is probably the hardest part—Twitter should introduce a per-tweet configurability for these settings.
That’s the kicker. Individual tweets could be deleted (fully gone), hidden (just for you), private (for your followers, no retweets), or public. That would be a big change! But it’s important.
(Another side note: when I posted this opinion on Twitter, I got a few responses that such an option should not be available for public figures or politicians or what-have-you. I tend to think that we should not rely on Twitter to preserve these public records, and that their introduction of a feature that has real benefits to real users shouldn’t be hindered by the possible exploitation by shifty officials. And just as with deleting tweets or blocking critics, the behavior can be barred without Twitter needing to intervene.)
Twitter the company has mismanaged Twitter the product in a dozen different ways, and its policies on speech, censorship, harassment, and abuse have often been incoherent. It’s also provided an invaluable platform for many of my favorite writers and thinkers, and done so in a way that is reasonably consistent with the technology of the web.4 Giving users more tools—in this case a new privacy setting with more granularity—is the right next step in overcoming the former without diminishing the latter.
By definition, most of the articles about deleting tweets are written by writers. Writers, again by definition, tend to write more stuff down. So maybe the benefits of having a place to write stuff down if you otherwise wouldn’t isn’t quite as clear to them. ↩
Two good things that stand out: canonical URLs for every tweet, and the fact that unavailable embedded tweets degrade gracefully into text. On the other hand, the normalization and ubiquitous use of a URL shortener sucks. ↩
One thing I enjoy is Norway’s public service broadcaster’s production of a train ride from Bergen to Oslo, which was first broadcast in real time, over seven or so hours, in 2009. It’s predictably pretty quiet stuff, but—at least now that it’s on Netflix—there are in fact subtitles of what little dialog there is.
Netflix makes it pretty straightforward to extract the subtitles from a given program, and it stores them according to a very fun standard called the Timed Text Markup Language, or TTML, which just missed adoption by the WHATWG in favor of a lighter-weight, less-XML spec called WebVTT.
Anyway, I pulled out the (very spare) subtitles in that format and wanted to convert them to something a little more usable. So first I converted them to JSON, and produced an array with an object for every subtitle, and then processed it a little further and created a version where “adjacent” subtitles are combined into single objects.
The result is nearly as hypnotic as the original video:
A little over a month ago, I launched one of the Special Projects I’ve been working on at my new job at the Freedom of the Press Foundation. The Threatened Outlets collection at Archive-It aims to capture the archives of news sites that we deem vulnerable to “the billionaire problem,” wherein wealthy individuals or organizations can eliminate unflattering coverage through litigation or by purchasing media companies altogether. From the launch announcement:
Those efforts help individual journalists. But another important thing we can do to reduce the effectiveness of this kind of attack on press freedom is to commit ourselves to the wholesale preservation of threatened sites.
In this case, we seek to reduce the “upside” for wealthy individuals and organizations who would eliminate embarrassing or unflattering coverage by purchasing outlets outright. In other words, we hope that sites that can’t simply be made to disappear will show some immunity to the billionaire problem.
To my surprise and delight, our launch received some nice and encouraging coverage, in WIRED, TechCrunch, The Next Web, The New York Times, and others. Sometimes when you’ve got your head down on something that feels like a niche topic, it’s nice to come up for air and realize that the general public is interested, too.
I had the opportunity to catch the new movie The Florida Project this week, without any background on how it was made or what it is about. That’s my preferred context for seeing movies, but it does sometimes lead to my feeling a bit adrift in the theater, or—in this case—stressed that something catastrophic was about to go down. Still, I feel like I end up with a better idea of what the movie is actually about if I don’t know the plot going into it. Here’s the trailer, if you want a sense of it.
That was certainly the case with The Florida Project, which has attracted nearly universally positive reviews from critics (and some confusion from audiences about the ending). Just about every review I’ve read since seeing the movie describes is as “empathetic,” which is spot on, but none really dive into what the audience is feeling empathy for. This post lays out my explanation, and to do so includes some very minor spoilers. I don’t really think this movie can be “spoiled,” but skip this post if you want to.
To my mind, what the movie does a great job of showing is how people seek spaces where they can be autonomous and in control. It shows children exploring and imagining, but (most importantly) testing out their agency. At the same time, it shows adults, living precarious existences, staking out small pieces of space where they are in charge.
The building manager Bobby is an obvious example, and his arguments with tenants can almost always be framed in this light: whether he’s enforcing the pool rules, confronting unwelcome strangers, or (most tellingly) moving tenants around to prevent them from establishing residency, he is exerting control over his designated space and limiting the autonomy of the people in it. In that last example, too, he alludes to the fact that he’s an agent of another level of power, specifically pointing at the surveillance cameras to explain how his hands are tied.
Halley’s actions in this area are even more illuminating. You can see it best when she gets into conflict with her best friend Ashley. Though Halley doesn’t even know the basis of their argument, she takes it out on her friend by finding a space where Ashley has no autonomy—her workplace—and exercises arbitrary control over her. From that perspective, it’s only a matter of degree when their dispute escalates to violence, and then when Ashley (it’s implied) calls for an intervention by the state.
In that penultimate scene, where Halley is arguing with DCF agents, she makes this unspoken theme somewhat explicit, saying that they’re on her “property.” Throughout the film there are a lot of reasons to think that unit 323 isn’t really Halley’s “property,” but in that moment she reaches for that language to express that she is in control.
The final scene takes that declaration and escalates it to a sort of surreal and metaphorical place. Echoing the film’s tagline (“Find your kingdom”), Moonee and her friend run to a place that feels like their kingdom, where they are the kings. It’s a bit jarring in a film that’s seemed so grounded and “real” to get such a visual metaphor in the closing moments, but it works (and at the same time drives home the juxtaposition between poverty and expensive fantasy that runs under the whole story).
Maybe most movies could be described in these terms—as Octavia Butler said, “all struggles are, essentially, power struggles.” But I think it brings a lot to The Florida Project in particular to think of areas of control and autonomy. The characters shown in the movie are living precarious existences, and seek that kind of control where they can get it. One message from the film is that, where these kinds of spaces aren’t really available, people will find a way to exert control nonetheless. The empathy in this movie is showing the decisions of people who have been rendered powerless in society.
Policies that perpetuate wealth inequality or pervasive surveillance make communities feel that powerlessness. When people talk about those communities “voting against their interests,” it’s worth considering that interest in exercising some kind of control.
