Technology

Twitter turns its back on open-source development


Elon Musk Twitter account with Twitter logo next to it

NurPhoto/Getty Images

When Elon Musk took over Twitter, he started by firing half of the social network’s staff. That’s a recipe for failure right there. Leaving aside Musk’s management and policy failures, social networks need experienced developers to keep moving forward. 

Ranting about the remaining employees needing to be “extremely hardcore” doesn’t help any. And, adding insult to injury, Musk has also abandoned Twitter’s open-source projects.

Like almost all modern software companies, Twitter depends on open-source programs. The network itself runs on CentOS 7. This free Red Hat Enterprise Linux (RHEL) clone comes to the end of its life at the end of June 2024. 

Also: The best Twitter alternatives

According to Twitter sources, the plan was to move to CentOS Stream. Now, there’s literally no one left to plan out, never mind oversee this operating system migration. 

But Twitter relies on open-source software for much more than its foundational operating system. As Will Norris, Twitter’s former open-source lead, told me in an exclusive interview, “I talked at length with [then Twitter CEO] Parag [Agrawal] about improving Twitter’s investment in critical open-source projects.”

He continued:

When I joined, there were already a number of large modernization efforts underway that had big open-source components. The Pants build system was being replaced by Bazel. Work was happening to prepare to eventually replace Apache Aurora and Mesos with Kubernetes. And, we were already some of the largest users of Apache Kafka, Hadoop, and Scala. We also had a custom fork of the JVM [Java Virtual Machine], which we hoped to eventually open source. There was a lot of amazing work happening, and they successfully hired really good people from those communities to work on those projects.
And then this year happened :-\

First, Musk began his on-again, off-again efforts to buy Twitter. Because of this, Twitter’s executives froze its open-source plans and investments until they saw a clear path forward. What happened instead was Musk firing all the executives. This was soon followed by most of the developers. Today, well over half of Twitter’s staff have either been pushed out or jumped out.

Also: Stop using Twitter to log in to other websites

Norris said, “Most of the key people that were working on open source at Twitter have left.  All of the engineers that I worked with on open source are gone.”

What that means for Twitter, Norris thinks, is:

In the short term, probably not a lot. It’s relatively easy for Twitter to shift back to just being a consumer of open source and not making any meaningful contributions back. That’s what loads of companies do anyway. They can continue using CentOS, Scala, Kafka, and all the rest, just like they have been. For projects that were in the middle of migrating, like Bazel and Kubernetes, that might be a bit more painful to halt, depending on what state things were left in. I have to imagine all of the focus is just keeping the service running and adding whatever product changes Musk wants.

Current Twitter employees tell me it’s all they can do to keep the wheels turning. I’m also told that the real reason for the delayed rollout of Twitter’s $8 monthly Blue service isn’t just because it was abused by users impersonating brands and high-profile user accounts, but because developers haven’t been able to automate a way to block such abuse. 

Also: Why Hive Social is not a viable Twitter alternative (yet)

That comes as no surprise. No one has ever successfully automated social networking moderation. It’s hard to imagine developers, who have been called in for late-night “program or be fired” work sessions, solving this intractable problem. 

As for the long term, Norris believes Twitter has become irrelevant in open-source communities. “They’ve lost all credibility as a serious engineering organization, I don’t care how much you call yourself “hardcore.” Open-source communities are built on relationships and trust, and now Twitter has neither with these groups. They’ve lost any ability to participate meaningfully in those communities.” 

But, there is one immediate problem: Twitter’s own open-source projects. Norris said: 

Many of them are no longer actively maintained (which is its own problem), and they’re quite popular, especially in the Scala world (Finagle, Twemoji, Scalding, and Algebird). Twitter had processes in place to keep some of those open-source projects, like Finagle up to date with the internal copies of the code, but none of that was fully automated. I highly doubt that there is anyone left to do that work. So what happens to the external users (which includes companies such as ING Bank, Pinterest, and SoundCloud) of those projects?

Nothing good. In addition, Norris continued, “Maintenance of projects in is going to be weird.” That’s because, “Twitter has an internal system to manage its presence on GitHub. It allows Twitter employees to register their own GitHub account to access their Twitter open-source projects. It used to be the case that when someone left the company, their access was left in place, but they were moved from being a ‘member’ of the Twitter org on GitHub to being an ‘outside collaborator.'” 

But even before Musk took over, Twitter sometimes failed to take developers who’d left the company off private Twitter GitHub projects. One employee still had access to Twitter’s GitHub and source code in August 2022 — 18 months after leaving the company.

Also: Why Twitter will fail shortly

Improper access to Twitter’s code was being fixed earlier this year, but that work was never finished. So, Norris said, “projects whose maintainers left Twitter years ago may still have interested people that continue to have access.”

Of course, the people who’ve recently been fired or quit and who “are the most knowledgeable about the remaining active projects, have almost certainly had their access removed.” So, it’s quite possible that ex-Twitter employees from months, or even years ago, still have code access, while the more recent ex-staffers, who knew who should have code access, are no longer around to fix things. 

As for Twitter’s own open-source projects, such as Finagle, Norris doesn’t expect “Twitter to do anything to continue maintaining these projects, at least not at the levels they were before. So realistically, all of those projects probably need to be forked and moved to a new home, but that’s going to be a messy process and potentially very disruptive.”

Open source, by its very nature, heals around damage. One way or the other, Twitter’s open-source projects will continue. But, Twitter itself, as a mere open-source consumer rather than an active open-source contributor, will benefit far less from its own projects. And this will be happening just as Twitter, under its new regime, needs all the programming help it can get.





Source link