Copyleft compliance misconception #2: Anyone can easily fix the incomplete source releases that companies provide

by Denver Gingerich on December 11, 2018

This post is part of a series on copyleft compliance misconceptions. For our previous post, see Copyleft compliance misconception #1.

As Conservancy's FLOSS License Compliance Engineer, I receive many reports of copyleft noncompliance every week, and the people reporting them are often rightly concerned that the compliance issues are not fixed quickly. These issues often arise on devices such as media players, Android phones, broadband routers, and even vehicles. In each case there is some copylefted software that a user wishes to inspect or modify (which is their right, according to both the license and the morality of software freedom) but they are unable to, due to noncompliance. Usually, those who understand software observe noncompliance as a real-world, practical problem of incomplete (or entirely missing) source code and build information.

In the course of resolving these copyleft noncompliance incidents, Conservancy normally receives a candidate source release (or "CCS (Complete Corresponding Source) candidate") from the company whose device or product is out of compliance. You might think that it is a straight-forward task for us to then to bring that CCS candidate into compliance with a few fixes. Unfortunately, that is normally not the case. There are a variety of reasons for that, which no one can resolve themselves without assistance from the software distributor. I've outlined the most common problems here in hopes that companies (and others) who wish to do right and comply correctly with copyleft licenses can make their own fixes earlier -- ideally before shipping a product at all, but at the very least, quickly at the time they are first contacted about a compliance problem. Learning about these issues and being prepared before a compliance problem is found can make a big difference! For example, clauses like GPLv3§8, the Linux Kernel Enforcement Statement, and Red Hat's Cure Commitment have a very short window of 30 days to fix violations. It's rare these days that companies get their CCS in order in 30 days after a violation is reported, but following the advice below could definitely change that!

We don't stand on ceremony with regard to copyleft compliance; our goal in any copyleft enforcement action is to bring whatever of our own (meager) resources to bear to aid a company in producing compliant CCS. In an ideal world, we'd just provide patches or a list of fixes when receive the first "round" (as we call it) of CCS candidates.

Trying to do this, in fact, is a big part of my job. When I evaluate a candidate source release for correctness, I normally create a list of suggested fixes (often including patches the company can programmatically apply) alongside the list of issues that I send to the company if I find the source release is incomplete. I then send these lists to the company in the hope that they will take our suggestions and fix their source release.

However, the result of sending the list of issues and associated fixes seldom results in the company responding with a source release that is complete and correct (especially in the case of a first "round" check). Over the years, I've noticed the causes for this generally tend to fall into one of these categories:

A. The company believes they don't need to provide source that is derivative of and/or has been combined with work licensed under a strong copyleft license like GPL

Thankfully, this situation is relatively rare, but it is probably one of the more public causes for incomplete source that people are aware of (see, for example, Christoph Hellwig's case against VMware that Conservancy is supporting and (partially) funding). In this case, Conservancy has notified the company that their source release is missing source code for certain binary files that are derivative of a copylefted work. Normally this comes up with driver modules for the Linux kernel -- the company does not wish to release (or does not have, because their upstream failed to provide) the source code for a driver they are shipping, but the driver is clearly combined with Linux. We generally try to find out who provided the driver to the company to begin with, and then follow up with them to try and obtain the source code.

In this situation we are naturally blocked from achieving complete corresponding source, as part of the source is clearly missing and the company is refusing to provide it to us. Short of reverse engineering their driver, this is no way for Conservancy or anyone else, other than the authors of the proprietary driver, to provide a fix for this situation.

The best approach for companies in this situation is to assure at the time they source a product that no proprietary Linux modules are involved. Often, downstream companies don't invest the resources to hire their own engineer (or a third-party engineering service organization) to rebuild their Linux-based firmwares from scratch. We strongly recommend anyone building Linux-based products do this, as problems like this can easily be found before you make a deal with an upstream vendor. While in this negotiating phase, the vendor will want your business so you'll have a lot more power to get the upstream vendor to provide CCS and even to indemnify you down the road if there turns out to be a problem, as described in D, below.

Solving this problem admittedly needs more lead time than any of the others. Fortunately, there are many (and more common) situations that come up, and while they don't generate impasse, a basic approach of "planning ahead" can go a long way to avoid repeated attempts at compliance.

B. The company applies one or two of the fixes then sends back a new known-incomplete candidate

One of the most common causes of a company not correcting its source release in the first "round" of CCS checks is simply that they did not implement all of the fixes we provided to them. This wastes both their time and Conservancy's, since anyone reviewing that "round" needs to figure out which fixes they implemented and which they didn't, and then send back a new report to them highlighting the fixes that were already provided in the last report. Then the company needs to go back and (hopefully) implement more of the fixes to send yet another candidate back to Conservancy.

Conservancy has tried to determine why this cause of incomplete source is so common, but unfortunately few past violators are able to explain clearly why this occurs. We suspect that the number of fixes that a company needs to make feels overwhelming to them so they simply don't bother with most of them. Or perhaps they think we won't notice that they only implemented a few of the fixes; we have seen situations where individuals who complain about CCS to a violator often run out of time and patience seeking corrected CCS, so perhaps these companies have misunderstood that incomplete CCS to be satisfactory merely because the individual never followed up. In any case, we hope that by highlighting this problem in blog posts like this, it might become less of a problem in the future.

C. The company fixes all the issues we highlighted, but there are new issues in their candidate

This is perhaps the next most common cause of a company not providing a complete source release on their second try, but is less insidious. An example of where this might happen is when a source release clearly requires a cross-compiler in order to build, but the company does not specify which one (and the code makes it difficult for Conservancy to guess, so Conservancy can't attempt to use a potential cross-compiler itself). When the company does respond with the name of the cross-compiler they used (or the source and binaries of the cross-compiler itself), Conservancy often then finds a number of issues with the rest of the source that cause the build to fail even when the correct cross-compiler is used.

There are a range of other reasons for this (such as a component that now builds revealing that a dependent component never built properly to begin with), but all result in the same outcome: additional rounds of CCS checking that cause Conservancy and the company to expend significantly more time on trying to resolve the problem. Naturally, this problem is exacerbated when combined with B above, which we see much more often than we'd like.

D. The company is as frustrated as we are, but has been intimidated by their upstream.

We have some degree of sympathy for many violators we encounter, because we know their upstream vendors are the bad actors and they've been caught in the middle. By the time an embedded Linux electronics product makes it to the mass market -- when a user might purchase it and report a violation -- the name-brand vendor has made various complex business deals with one or more upstream vendors. Those upstream vendors often knowingly violated the GPL on Linux, but do not inform their customers about the presence of Linux, or the incompleteness of the firmware sources. Most downstream vendors never rebuild the main firmware; they develop a user-space application on top of it quickly and push it to market. When the violation is discovered, the main vendor is in a very weak negotiating position, and fears problems for future products. Many violators have lamented to us this difficult problem, but are confused about what to do about -- they simply become stuck in the middle sending us CCS candidates from an upstream vendor who has already decided to violate the GPL knowingly and willfully.

Conservancy has tried many creative solutions to this problem, including asking the violator to join us in informal or formal legal action against the bad actor. Sadly, few companies care enough about the GPL to take this stand, and we're often left with incomplete CCS and an indefinitely continuing GPL violation. This is what occurred with Tesla, which is why we applauded them for going public with this as they try to address their ongoing GPL violations of incomplete source.

Conclusion

It is extremely rare for me to find that a company has fixed all of the problems in their source release on the first attempt, even when I provide a complete list of all the problems and patches to fix each of them. That's why I wanted to highlight some of the causes for this that we normally see, both so that people using copylefted software in devices they've bought know what we're up against, and so that companies are aware of the common pitfalls and can correct their source releases accordingly.

[permalink]

Tags: conservancy

Keith Packard: Inspired & Inspiring

by Deb Nicholson on December 4, 2018

This is part of our ongoing series to highlight our generous matching donors. Keith Packard has been working in free software for more than thirty years and is a long-time contributor to Debian, X Windows and more recently the Linux graphics driver stack. Keith and several other outstanding individuals are joining Private Internet Access and a big anonymous donor in offering a total of $90K in matching funds to Conservancy for our continued work to provide both a "back-office" for free software and a clear voice in favor of community-driven licensing and governance practices. You can join him and donate today!

Deb: What's the best recent trend you've seen in free software?

Keith: I've been excited to see how rapidly the free software communities I participate in have adopted strong codes of conduct. I feel like the tone of communication has become kinder and accepting. We've still got a long ways to go, but it feels like we've at least accepted that this change is necessary and good.

Deb: What do you think non-profits are uniquely positioned to accomplish in free software?

Keith: I'm afraid I'll have to be very US-specific with my answer as the US tax code colors how people within non-profit organizations work, even non profit organizations with an international scope.

The free software non-profit organizations that I am involved in have operated under the 501(c)3 rules for public charities. These rules are designed for groups focused on charitable, scientific, educational and certain other purposes and require a strong commitment to the public good. For instance, X.org was able to operate under 501(c)3 rules only after demonstrating (to the US government) a many-year history of providing direct educational assistance to new and existing software developers through our conferences and other outreach programs.

The goals of free software include allowing all people to control their computing environment and allowing people to teach, learn and share all of the software they use. An organization supporting free software for the public good is the best way I know of to defend and promote the essential software freedoms.

Deb: Is there a free software project that you wish existed but (as far as you know) no one has started working on?

Keith: One of my little quirks is that, even in 2018, I run my network infrastructure out of my own home (mostly email, git and web pages). I feel this offers me a better legal framework should someone want to seize any of my data, plus it keeps me from being subject to any corporate policies I might not like.

My web site is published using ikiwiki, which works pretty well but is both overly complicated and missing some key features. I like the general plan -- write markdown, commit to git and publish the resulting web site. I don't like the markdown it uses; it's old and written in perl. I don't like how publication works -- you push source code to a git repository on the web server where a git hook runs to compile that source code into your web site.

I recently packaged cmark-gfm for debian; that's the common mark to HTML conversion program that github uses for all of the markdown content that people publish there. It's fast and self-contained, but it doesn't generate whole web pages, just HTML fragments.

What I'm looking for is a self-contained web site publication program that takes a collection of templates and a collection of common mark files and generates a whole web site from that on my laptop. Once the web site is complete and tested, I should be able to copy it to the web server unchanged where it can be served directly.

Essentially, I'm interested in free software to help make running a simple web presence easier, more secure, and still look good so that people aren't stuck using commercial services where users are the product.

Deb: What do you hope to see Conservancy accomplish in the next five years?

Keith: Of course, I'm hopeful that Conservancy will continue to grow and spread the good news about free software. I expect to see many more member projects join. I'd love to see parallel organizations form supporting freedom in other areas of the arts and sciences -- everything from computer hardware to clothing design. I think those communities can learn much from what Conservancy has accomplished, and I hope we can learn from them as well.

Deb: Anything else you'd like to add?

Keith: I'm so pleased to have been asked by Molly to participate in this fundraising drive; her vision of what we can do as a community is inspiring to all of us.

Please take a minute to help Conservancy continue to spread the good news about free software and meet this exciting year-end match, today!

The picture of Keith with his cat is a selfie and is available under the CC BY-SA 4.0 license.

[permalink]

Tags: conservancy

Molly de Blanc: Free Software Superstar

by Deb Nicholson on November 27, 2018

I recently interviewed the inestimable Molly de Blanc. Molly is on the Board of Directors for the Open Source Initiative, works as a Campaigns Manager for the Free Software Foundation and also happens to be an amazing baker. She has been working in free software for 4 years, and involved for 10 years -- plus she is the driving force behind the individual super-donor part of our year-end donation match. Molly and several other outstanding individuals are joining Private Internet Access and a big anonymous donor in offering a total of $90K in matching funds to Conservancy for our continued work to provide both a "back-office" for free software and a clear voice in favor of community-driven licensing and governance practices.

Deb: I know you're passionate about free software and an enthusiastic supporter of non-profit work. Can you tell me why you think non-profits are important for free software?

Molly: Honestly, I think we just can't trust corporate actors to do the right thing for user freedom without financial incentives. Nonprofits are powered by their donors, members, and supporters. This means that they're working for those people -- people who care -- rather than typical corporate and financial interests. When you have a nonprofit working on a cause, you have someone working for that cause.

Creating good free and open source software and infrastructure isn't enough. We need organizational support for those projects. We also need organized advocacy, which best comes from nonprofits and foundations, whose entire mission is supporting user freedom.

Deb: How did MollyGive start? Who named it?

Molly: For a little background context, starting somewhere around 2013 or 2014 I began to offer to match people's end of year donations from my own donation fund. I don't remember the exact year, but my blog references it "happened again" in 2015. I forget what I called it for the first year/s, but David Nusinow dubbed it MollyGive, which was catchy if not a little egotistical when I need to refer to it.

I liked donating, but had trouble deciding where to most effectively donate. I wanted to reach out to causes that mattered, but were outside of the sphere of my volunteer work, which at the time was around free software, libraries, and music. I have amazing people in my life, and trust them. It was nice to contract out my decision making to others. I've gotten to reach groups like the American Indian College Fund, Black and Pink, and MassCare. (Check out the complete list here.)

I also wanted to encourage others to give. I started donating when I didn't have a lot of money -- and understand how demoralizing it can feel when you're only able to give $5. I hoped to encourage people to give by helping them to realize how much their giving matters.

It's important to note that I don't consider this an extension of MollyGive -- for me it is how I'm using a large chunk of the MollyGive funds this year. [We] didn't explicitly ask others to participate in MollyGive -- we asked them to help Conservancy.

Deb: How hard was it to find folks to join you on this match after you had the idea?

Molly: Well, Karen helped me! The first few donors were easy to find. Between Linux Conf Australia 2018 and DebConf 18, I picked up a few enthusiastic individuals. After that, I got a lot of "nos." Karen and I together helped them change their minds -- that's how we got two of our matchers. At least one person approached us after hearing about it, which was really inspiring.

Deb: What do you hope to see Conservancy accomplish in the next five years?

Molly: Conservancy is slowly gathering all of my favorite free software projects (that aren't already 501(c)3s) under its umbrella, like the Debian Copyright Aggregation Project, Etherpad, Git, Outreachy, and Reproducible Builds. I expect that in the next five years you'll get the rest.

As someone who has worked at nonprofits for most of their career, I'd say I probably think about the Conservancy differently. For example, I'd like to see the staff expanded, especially to include more organizational infrastructure roles, like a development director. It would be great if it hosted a summit for its member projects.

From a programmatic perspective, I'd really like to see others recognize the Conservancy's role on the cutting edge of working for user freedom. This includes things like being recognized by media outlets as an authority on issues relating to ethics in technology; providing support for developers and engineers looking to build freedom respecting contracts at their work places; and increased copyleft compliance. One of my hopes is to see strong legal wins for copyleft, and I think the Conservancy will play an integral role in that happening.

I see Conservancy as the organization that can do what no other nonprofit can do for software freedom. I see a deep understanding across its entire staff of the necessity of free and open source software. I think that you and I have similar visions for a world where user freedom is built into every piece of technology. I look forward to seeing how the Conservancy helps turn that vision into reality.

Deb: Anything else you'd like to add?

Please donate! I love giving away money -- especially other people's! By donating you're not just helping free and open source software, you're helping me, personally, because I get to have another successful year of matching donations.

I'd like to express my deep gratitude to the other people participating in the match -- whether they're choosing to be public about their identity or staying anonymous. I'd also like to express my gratitude to you, reading this, and everyone else who is choosing to support Conservancy.

You can read more of Molly's thoughts on the Conservancy match this year on her personal blog.The photo of Molly is is CC-BY-SA 3.0 by Luc Rocher.

[permalink]

Tags: conservancy

Thoughts on IBM’s acquisition of Red Hat

by Karen Sandler on October 31, 2018

There’s been quite a stir in our communities following the announcement that IBM is acquiring Red Hat. As I considered the announcement, one part of the email to employees by Jim Whitehurst posted on the Red Hat blog really struck me:

I appreciate that everyone will experience a range of emotions as a result of this news. Excited, anxious, surprised, fear of the unknown, including new challenges and working relationships - these are all ways I would describe my emotions. What I know is that we will continue to focus on growing our culture as part of a new organization. We will continue to focus on the success of our customers. We will continue to nurture our relationships with partners. Collaboration, transparency, participation, and meritocracy - these values make us Red Hat and they are not changing. In fact, I hope we will help bring this culture across all of IBM.

In addition to the normal anxiety, surprise and fear experienced by employees of companies in the wake of an announcement of a merger, takeover or ordinary reorganization, this transaction will also reverberate through the community outside of the company. Free software contributors across many communities and industries are feeling some of the same apprehension and unease that ordinarily would be reserved for employees.

I wish IBM and Red Hat luck, and I’m optimistic that the partnership will yield good things for both companies and their employees. I hope that following the acquisition, Red Hat is able to maintain its special relationship to the free and open source communities it shepherds, and that its employees continue to feel empowered to support critical free software solutions in a community-focused way. I also hope that in its announcement to keep Red Hat its own unit within IBM is an indication of IBM’s support of Red Hat’s unique business and that the deal does wind up bringing that culture to more of IBM. While some folks at IBM are important contributors to free software, IBM’s is primarily a culture of proprietary software and Red Hat’s is one of open source, so in my view this solution is likely to yield the most success anyway.

I’ve heard people imagining the best from this deal, and also people imagining the worst. The one thing everyone can agree on is that there’s a lot of uncertainty, despite whatever reassurances are contained in corporate messaging. Because of this, I think it’s a good time to remind everyone of the ways we can protect ourselves now and in the future from these kinds of uncertainties related to changes in ownership, structure or motivations of corporate players in free and open source software:

• Use copyleft. Quite a lot of the software projects that Red Hat plays a critical role in are licensed under a version of the GPL. When we use strong copyleft we set the ground rules for corporate actors to participate with each other and with the public. We get a level playing field and assurance that companies will be less incentivized to go their own way. (We also get other good benefits like the right to the source code, allowing us to be in control of the technology we rely on.)

• Support strong charities. Nonprofits, and in particular charitable nonprofits, keep the community’s interests at the forefront. They can serve as copyright aggregators in a more trusted way, facilitate cooperation of different stakeholders and function in a variety of ways to forward the long term interest of software freedom. The more we invest in our critical foundations, the less vulnerable we are to changes in corporate actors. The stronger foundations like GNOME, Conservancy and the FSF are, the easier it is for communities to weather a new direction from a prominent company.

• Encourage diversely held interests. Making sure that interests are not aggregated in single for-profit actors insulates communities against a change in ownership of a company. For effective success in using copyleft, copyrights must not only be with for-profit companies but have substantial copyright holding from charities and individuals. Also, technical leadership should include actors from different types of entities. When copyrights are held by many actors in the field (or by charitable nonprofits), it’s much harder to relicense projects as proprietary or on otherwise less ideal terms, and copyleft enforcement is a community-driven rather than for-profit activity. When care of the technical direction of a project isn’t significantly concentrated in one company, free software projects are more robust. Development may be slower with community-led contribution, but we can have greater confidence about the stability of the project and the community.

The interests of companies are not always aligned with the free software community or the public. Companies that seem to be in one stable condition today may change dramatically tomorrow. While I expect Red Hat to flourish under IBM ownership, the acquisition is a good example of the kinds of changes we must be prepared for down the road, whether it be with Red Hat or any of the other companies on which we’ve come to rely.

[permalink]

Tags: conservancy, GPL