In 2005, I was in the thick of things. I worked as an independent performance consultant, and Sun Microsystems just released DTrace, a product that can instrument any software. He gave performance analysts a “x-ray vision” function. While writing and publishing advanced performance research tools based on DTrace (my open source DTraceToolkit and others DTrace tools, that is, scripts), I noticed something strange: I published more DTrace tools than Sun itself released. Probably there is some kind of internal project in which all the people who know DTrace are busy?
DTraceToolkit v0.96 (2006)
Since I was not an employee of Sun Microsystems, I was not privy to internal company projects. However, I did training and consulting for Sun, helping the company’s clients with system administration and performance. Sometimes Sun would invite me to their client meetings and other events that would be of interest to me as a local professional. Then I lived in Sydney (Australia).
Once I was told that a very important person was coming from the USA, with whom it would be interesting for me to meet. I can’t remember the name, but I was told that this is a DTrace specialist and developer from Sun who is flying around the world talking about a new Sun product based on DTrace. Aha! Probably the same internal project!
But this should be an extraordinary project. I’ve seen some of Sun’s awesome tech, but I’ve never heard of a developer traveling on a world tour. It will be super cool and most likely leaves no stone unturned from my previous DTrace projects.
This VIP was returning for a few days in Sydney before traveling to the next Australian city, so we arranged to meet at Sun’s Sydney office.
The DTrace technician arrived in a standard business suit and had a strong American accent in his voice. It was evident that he was a little tired of the world tour. He just flew in from South Africa and New Zealand, and listed other countries and cities to which he will travel next. Two more Australian Sun employees joined our meeting. One of them introduced me:
“Brendan teaches courses with us and he has done DTrace projects.”
Such a humble view is the norm in Australia (especially for Australians), and I wondered if he understood this cultural difference. Another difference was that in 2005 there were quite a few engineering positions in Australia, unlike in the United States. For example, all Sun Microsystems Australia employees were involved in support, but not development, and other tech giants had yet to hit the market. That is, in those days in Australia, amazing engineers grabbed any job available.
I tried to explain in more detail what is meant by “worked” by saying that I wrote DTraceToolkit, but he was not impressed. He didn’t know my name and had never heard of DTraceToolkit. For him, I was just a random person.
However, he kindly agreed to show me a small demo. His DTrace-based product was an add-on to a large Sun GUI that I was already familiar with. Once downloaded, it showed you how to launch one of several DTrace tools by double-clicking the icon. The tool could either display raw data in a separate window, or display the results as a line graph. It seemed pretty mediocre… The GUI already had this functionality and I was hoping for new GUI features.
The only novelty was the instruments themselves, of which there were quite a few. He briefly talked about the amazing new surveillance capabilities they provide (most likely, he has said this many times in an effort to impress clients). I got the feeling that he didn’t expect me to fully appreciate their value.
But I actually I knew about these tools because I was coding similar functionality for my DTraceToolkit. They were helpful … but I was expecting a whole waterfall of amazing new DTrace functions.
“I’ve done this before, I’ve written the tools myself that perform these functions!”
“Well, yes, of course,” he didn’t answer that way, but his gaze was eloquent — he didn’t believe me and didn’t think that I was fully aware of their necessity. It was a major innovation worth billions of dollars from US multinational Sun Microsystems. And I was just some ordinary Australian.
I looked among the GUI icons for something new for myself, and the most interesting was the tool for tracing socket I / O. I tried working with it in 2004 (socketsnoop.d) and published the resulting result in open source, but my tool was imperfect: I did not have full access to the source code of the kernel, so I had a hard time figuring it out on my own by analyzing the “black box”. It worked with most types of TCP traffic, but not with others, which I reported in the comments to the script. Also, I didn’t add it to the DTraceToolkit because I didn’t think the tool was complete. Therefore, of all his instruments, this one was the most interesting for me. Sun could have done a lot better because it had the source code and finished the tool.
“Can I see the socket I / O script?” I opened the terminal. At first, the developer looked preoccupied, as if I shouldn’t be allowed to peek behind the scenes, but then he realized that this was another selling point of the product: “Yes, of course, you can even add new tools to the GUI!” After a pause, he added: “If, of course, you have them.” I have them, that’s for sure. He told me the path to look for, and after a while I got to the folder with all the tools he was demonstrating.
All instruments had familiar names. One was even called socketsnoop.d. Suspicions arose in me.
Can not be.
I have printed socketsnoop.d. Appeared on the screen my own script… This was the same unfinished attempt that I made a year earlier and published as open source. It contained weird code that seemed logical to me at the time (using PFORMAT before defaultargs) and it was written in my old style. I read my script…
“This is MY script.”
I opened a few more tools, and I saw the same everywhere – they all were mine… This awesome new Sun product that Mr. VIP has been promoting around the world was actually my own open source tools.
My jaw dropped. But he didn’t seem to believe me.
You can’t do this
Using grep, I searched all the tools for my name, which was indicated in the first comment to all of my tools, to prove that they are mine. But I didn’t find anything. My name was removed.
Some of the tools even had the line:
# Author: Brendan Gregg [Sydney, Australia]
And now this man has come to Sydney and is trying to sell Brendan Gregg’s tools to Brendan Gregg.
One of Sun’s Australian employees interrupted me, “It’s copyright Sun Microsystems everywhere.” Most of my tools included my copyright and GPLv2 or CDDL license. But these only contained the standard Sun copyright message, and the open source licenses were cut.
“You removed my name! And copyrights with licenses! “
The second Australian told the guest: “So do it is impossible“. Silence reigned in the room, everyone present tried to comprehend what had happened. While some people at Sun motivated developers to contribute to open source and build a community, others robbed the community. They took his work, changed licenses and copyrights, and then sold it.
The guest was not ready for this and looked puzzled. He remained laconic, only said that he did not understand how this happened, that he could have already been given the instruments in this form (read: do not blame me). It seemed that he only partially believed our words.
The meeting ended quickly. I suggested that he grab newer copies of my tools directly from DTraceToolkit, because those older versions from my home page were already out of date and I have fixed several bugs in them since then. I also reminded him to leave my name, copyright and license in all of them.
In his defense, I can say that the meeting could have gone differently if I had been introduced in more detail. This is a cultural issue in Australia (keep your head down). To an Australian, representing an American-style person may sound like a boast, but it can also be useful as a quick way to communicate a person’s specialization.
I couldn’t believe that out of all the open source tools I published, socketsnoop.d was added to the product. This program was crude. Later, I wrote much better tools for working with sockets (in my books about DTrace and BPF).
A few years later, Apple added a lot of my tools to OS X. The company kept my name, copyright and open-source CDDL license there, and even improved some of them. Years later, Oracle did the same for Oracle Solaris 11, and the BSD community did the same for FreeBSD. I thank all these people.
You might say that it wasn’t Sun who did it, but some single, careless person. But there was something about Sun culture that encouraged this kind of negligence. My fellow consultants and I have experienced this before: the belief that Sun and only Sun can make the best use of its own technology, and everything outside of Sun was rubbish. When these Sun employees found something good, they had a tendency to think that it was created by Sun, which means it can be safely used and re-branded (and change the license), because they believed that the company already owns the copyright.
There were other people at Sun who were eager to do the right thing with me and my work. At least four times, my DTraceToolkit was embedded in software monitoring products without stripping the licenses. (In one case, they wanted to re-license it under the GPL, and we discussed the situation with Sun’s legal department, but that’s another story.)
By the way, then it was not the last time they inadvertently tried to sell my work to me, this was only the first time. I learned not to tell the salespeople that I created what they were trying to sell me, because then they started looking at me like I was crazy. Instead, I simply said, “I have a lot of experience with this technology,” and I limited myself to that.
I remembered this first case because my BPF tools are now starting to appear in software surveillance products, and over time will become much larger than the DTrace tools. In future posts I will write about this in more detail, but for now I advise developers the following: please do not rewrite my BPF tools and bcc libraries; try to build on top of them (in bcc python or bcc libbpf-tool versions) and get regular updates. The reason is that these are still under development products, and rewriting (forking) them will result in a dissipation of development efforts and force your customers to use outdated versions.
(With my software for creating flame-graphs, the situation is different: since this is a simple and ready-made algorithm that does not require special support, I do not see a particular problem in having others rewrite it. However, it will not hurt your gratitude, as I expressed thanks themeswho inspired me to create flame graphs.)
As for the “incredible demo”: it was not the awesome DTrace-based product that I expected after hearing about the world tour. In fact, they turned out to be my own instruments. I suspect that many open source developers will sooner or later find out that their code is being rebranded. But in my case, the circumstances were rather unusual. A US developer went on a world tour with software that he didn’t write, including inadvertently giving a presentation and demonstration to the author. He didn’t even seem to thank me.
Powerful VDS with DDoS protection and the latest hardware. All this is about our epic servers… Create your own plan in a couple of clicks, the maximum configuration is 128 CPU cores, 512 GB RAM, 4000 GB NVMe.
Subscribe to our chat in Telegram…