New CPUs, GPUs, and faster migrations: QEMU looks forward to 2017

by Will Hawkins on December 27, 2016

This series covers new developments and exciting projects taken on by Conservancy member projects. To learn more about Conservancy member projects, or the non-profit infrastructure support and services offered by the Conservancy, check out Conservancy’s Projects page. Please support Conservancy so we can continue to help all this important software.

The cloud—the great modern technology buzzword. Even those who don’t think of themselves as technical users have heard the phrase and perhaps even benefited from it. Though there are many proprietary cloud providers, OpenStack is the most popular FLOSS cloud software platform, powering massive web sites like Overstock.com and PayPal. What you might not know is that Conservancy member project QEMU is at the heart of OpenStack, and the project is proud to support them.

QEMU is a FLOSS project that makes it possible to emulate one hardware platform on another hardware platform and/or run multiple virtual machines (VMs) on a single physical machine. QEMU is just one of the many great FLOSS communities that Conservancy supports and I was lucky enough to be able to interview several of QEMU’s main contributors to ask them about their project, its future, and how Conservancy supporters have helped them succeed! In my interview with Stefan Hajnoczi, one of several QEMU subsystem maintainers and a contributor to the project since 2010, he said that the project benefits from Conservancy’s infrastructure, legal and community support.

An important moment in the life of any FLOSS project is when it adopts a structure that can outlast any single individual. Stefan says that Conservancy has helped QEMU make that trasition. Conservancy provides the infrastructure for holding domain names, hosting the project’s website, handling the project’s finances and accepting tax-free donations.

Conservancy also helps QEMU when the rare legal issue arises. “It’s difficult for any open source project that doesn’t have lots of funds to get legal clarity,” says Mr. Hajnoczi, and QEMU’s many different uses make legal clarity particularly important for the project.

QEMU is a widely used project and accepts contributions from a variety of sources, from corporate developers to hobbyists. Corporate and FLOSS projects of all kinds integrate and modify QEMU because its utility and flexibility make it a great foundation on which to build solutions for their end users’ problems. This means that QEMU is often mixed with software distributed under several different licenses. Because so many end users benefit from QEMU’s integration in these solutions, there are plenty of people who can report potential license violations that QEMU and Conservancy work together to resolve.

Although it’s already an invaluable resource in the corporate world, in other FLOSS communities and for many end users, the QEMU project is not slowing down! 2017 is shaping up to be a very productive year for QEMU and it could not sustain its growth without support for the user and developer community by Conservancy.

In 2017, QEMU will advance their support for the ARM and RISC-V architectures. Full support for these architectures is vital. The heart of almost every mobile phone is an ARM processor, and the chip is even starting to be used in datacenter servers because of its power efficiency. RISC-V is a completely open architecture specification developed by a consortium whose members include Google, Microsoft, Nvidia, IBM, and HP Enterprises, among others. The goal is to develop RISC-V to work in a variety of contexts, from high-performance computing to computer science and engineering education.

In 2017 QEMU also plans improvements in the software’s ability to move running systems between different computers without pausing execution, called live migration. QEMU has supported live migration since 2010 but plans on expanding support for this feature in the new year. This work will make it possible for administrators to immediately shift a VM to another physical machine without having to wait for the VM’s utilization to reach a certain level, a limitation that exists today.

Finally, 2017 will also bring new support for QEMU’s ability to virtualize graphics processing units (GPUs). These days many artificial intelligence and machine learning software tools are being written to take advantage of GPUs. Virtualizing those resources in the way that QEMU already virtualizes a CPU, hard drive or network card would reduce the total amount physical resources required for GPU-intensive applications by sharing the resources efficiently.

These advances are all driven by QEMU’s community of developers and users. Conservancy helps QEMU foster that community by providing hardware and software resources for Internet hosting and facilitating the nuts and bolts of its participation in Google Summer of Code and Outreachy. The work from developers mentored through those projects has pushed QEMU into new areas. Conservancy has worked with QEMU to make it as easy as possible for both mentors and mentees to work together productively.

Since its founding in 2004, QEMU has made a huge technical and social impact thanks to its role in facilitating cloud deployments. Its incredible success so far is only overshadowed by its future. Conservancy looks forward to continuing to work with QEMU as it expands and grows in 2017 and beyond.

[permalink]

Tags: conservancy, Year In Review 2016, Member Projects, QEMU

Report from the 2016 Reproducible Builds Summit

by Brett Smith on December 26, 2016

A couple of weeks ago I was at the Reproducible Builds Summit in Berlin. Over sixty representatives from all kinds of projects came together for three days to share information and ideas, plan solutions, and even squeeze in a little time to hack. It was my first real opportunity to dive into this work. I learned a ton, even enough to chip in a little, and I’m looking forward to working more on reproducible builds from here on out.

When we talk about reproducible builds, what we mean is a build process that produces the exact same binary every time you run it with the exact same inputs (like source code versions and compiler settings). If you’re interested in the details, check out the definition on the Reproducible Builds site—a bunch of folks hammered that out during the Summit.

You might think most build processes would be reproducible most of the time, but often the binaries include small inputs that are hard to reproduce, such as timestamps or build paths. Much of the work toward reproducible builds so far has focused on improving the inputs: removing inputs that aren’t really necessary to the final product, and better recording the ones that are. Once that’s done, most build processes are as reproducible as you’d expect. There’s still more to do there, but there’s enough of a foundation that we can start seeing some benefits from reproducible builds. Many of the discussions at the Summit were about planning those next steps.

Conservancy is really excited to help reproducible builds. Having a clear and trusted link from source code to binary helps the community in many different ways:

• The most obvious is security. When builds are reproducible, everyone can check for themselves that binaries they download actually come from the expected source code. We can demonstrate that unwanted code isn’t being added to distributors’ binaries, either accidentally or maliciously.
• A reproducible build is a documented build. When everyone can see exactly what inputs and build steps generated a binary, everyone can review and comment on that build process. It becomes easier to find binaries with “bad” inputs (like a version of a library with a critical bug) and plan an upgrade process for them.
• Reproducible builds can make license compliance easier for binary distributors. When a free software license requires distributors to provide source code, sometimes it can take a little work for them to figure out exactly what the right source code is. For example, if they have three versions of a development library installed on their build system, how do they know for sure which one went into the binary and should be included in the source code release? Reproducible builds record the answer unambiguously, in a format that can make it simple to put all the source code together.

We’ll reap the most benefits if there’s support at every level of the stack. Debian kickstarted the reproducible builds effort, and at the Summit there was a lot of great discussion about reaching out to other communities. Right now the focus is on other package distributors, so it was great to see representatives from Fedora, openSUSE, F-Droid, and Nix there. But our discussions also recognized the need for outreach to other projects that can play a role in this work, like build tools and other software that generates binaries that get shipped to users (such as filesystems or bytecode compilers). If you’re involved in a project like that, I encourage you to join us on the general mailing list for reproducible builds and introduce yourself. The more people working on this, the merrier!

Many thanks to all the Summit organizers for planning and running a productive working space. I’m already looking forward to the next reproducible builds meeting.

[permalink]

Tags: conservancy, Reproducible Builds

Chromium's Alice Boxhall Explains Why She Supports Conservancy

by Brett Smith on December 20, 2016

Alice Boxhall helps develop Chromium, with a focus on accessibility features. In this video, she talks about some of her favorite Conservancy member projects and why she supports the organization. Do you want free software to be for everyone too? Support Conservancy today!

[permalink]

Tags: conservancy, supporter, Member Projects

Catching up with Godot: An interview with Juan Linietsky

by Christine Lemmer-Webber on December 13, 2016

Godot is a multi-platform engine for 2D and 3D games. It provides common functionality for graphics, audio, physics, and more, so game developers can build on top of a solid foundation and spend more time working on the unique parts of their game. Godot is a Conservancy member project. (Become a Conservancy Supporter!)

Christopher Allan Webber, co-maintainer of GNU MediaGoblin and a Conservancy volunteer, recently sat down to learn more about Godot with one of the core developers, Juan Linietsky. Recently Linietsky’s been working on a new graphics renderer for Godot with funding from a Mozilla Open Source Support grant that Conservancy helps administer for the team.

CW: Tell us a bit about your project. What’s Godot, and why might a potentially-interested user pick it up?

JL: To explain why Godot is important, I believe it’s enough to explain some numbers about the software industry. Globally, the software industry moves around 400 billion dollars in revenue, out of which 100 billion are from the video game industry. Games are a huge industry that keeps growing.

Game development happens mostly with proprietary software, and almost half of it is done with a single proprietary tool (Unity 3D). For being such a huge industry, there are no comparable open source alternatives like Apache or Linux that game developers can use instead of proprietary solutions, even if they would be very welcome.

Godot is a project that aims to change that, by being a comparable open source offering. It’s growing very quickly in community and has a large and dedicated group of users and developers.

CW: So you’re a Conservancy member project! What was the decision like to join under the Conservancy umbrella?

JL: Godot is a project that has many users, institutions, companies and other non profit organizations interested in funding its development. As we the developers are only individuals, we need to make sure our donors and sponsors have a strong promise that their donations will be used for the benefit of the project. Conservancy is an institution with a flawless track record in this regard.

CW: Godot has its own internal language, I believe partly to control issues like garbage collection pauses. But Godot recently started adding support for C♯, which is garbage collected. Could you talk about your history of game scripting support, its motivations, and where you see it going in the future?

JL: The reasons why we ended up using our own scripting language are not obvious, but are very strong. Popular languages like Python and Lua were not designed with real-time applications in mind. The garbage collector can cause random stalls, and support for multithreading is not so efficient because they use giant locks. On top of that, the complexity of binding them to Godot was huge and very error-prone.

GDscript is a language that “just works” and Godot users love that. That said, we’re adding C♯ support due to popular demand, and the fact that Mono was released under the permissive MIT license a few months ago.

CW: How would you describe Godot’s community, and your relationship with it? Is there a direction you’d like to go towards stewarding community interactions?

JL: The Godot community is awesome, they are very helpful with each other (and if they are not, we make sure they are). We try to help everyone, no matter how much (or little) experience they have, and always encourage users to help each other.

CW: If someone is interested in getting into game engine development, with Godot or otherwise, where should they start and what sorts of resources are available to them?

JL: There is some documentation on the C++ side of things, and the code in general is well-organized. The best approach to starting with development is to implement your own feature (make sure main devs would like to merge it first) and then do a pull request.

CW: How long until Godot overtakes Unity? ;) More seriously, do you see Godot as an alternative to proprietary engines like Unity or the Unreal Engine? Or do you see Godot as serving a different niche?

JL: Our development is very focused on the following premises:

• Godot must be very easy to use, simple and to the point, even by compromising flexibility if necessary.

• It must be easy to make a 2D or 3D game look great with little effort.

So, in pratice, we aim for an engine that is easier to use than Unity, and that looks better than Unreal, and we are very close to getting there in the upcoming Godot 3.0.

CW: What kind of overlap do you have or hope to have with related libre software media production projects and communities like Blender?

JL: Many Godot users create their 3D art on Blender, and we’d like to have a better relationship with the team. We just heard they are working on a PBR viewport, so we will work hard to allow users exporting their 3D content as faithfully as possible to Godot.

CW: Thanks for the interview! Anything else you’d like readers to know about Godot before we wrap up?

JL: We always listen to our users, their problems and wishes even though we can’t always take immediate action. Currently we are focused on shipping a 3.0, wich will come with a completely new renderer. Godot 3.1 will improve some peripheral areas like the physics engine and the audio engine. Afterwards, I dream of a time where no new large features need to be implemented, so we can polish and improve as much as possible everything that is there!

Help Conservancy provide the stewardship to support projects like Godot by joining as a Conservancy Supporter today!

[permalink]

Tags: conservancy, Member Projects, Godot