Dealing with different development environments

Question

In corporate software development, all developers are, typically, using the same IDE running on the same version of the same operating system. In OSS, the team is distributed and providing their own development environment. This can lead to some difficult situations where one contributor checking in code breaks the build for another. I've personally struggled with this exact situation. Every pull from the main branch breaks my build configuration. I fix it, do some work, push back, and then my partner does the same, fixing his build configuration before he can get any actual work done.

What techniques/tools can be used to compensate for different development environments? Are there specific things that should (or shouldn't) be checked into the repository to help alleviate this?

Answer 1

In corporate software development, all developers are, typically, using the same IDE running on the same version of the same operating system

This is a misconception. Example:

I run Eclipse Luna with openjdk-1.7 on Ubuntu 15.04, my colleague runs Eclipse Kepler with oracle jdk 1.7 (previously 1.6) on Windows.

Still we got our builds working.

The goal is to capture the bare minimum configuration for things to work "out of the box", and discard everything else. There's simple tools for that in many many languages, the usual suspects are:

• Dependency Management:
  This is the tool of choice to manage what you need for development, building and execution. It grabs all the things you need from a repository somewhere on the internet and makes them available to your IDE and the build-manager. e.g. Ivy, NuGet, npm, Gradle, pip, ...
• Build Management: e.g Maven, Gradle, Ant (sorry I don't do much aside from java)

They provide often rather simple configuration files based on a declarative syntax (XML, YML, JSON) that set up your dependencies.

It's important to note that this does not include IDE settings!
IDE Settings are one of the things I'd never check in to source control. Nobody should have to care where the project is checked out. Nobody should have to deal with suddenly different syntax-highlighting and IDE extensions. The only viable IDE setting to version is Formatting preferences! Everything else should (and in 99.9% of cases can) be left to each developer themselves

Another important factor is making code not rely on platform features:

• Avoid hardcoded paths like the plague
• Prefer configuration over code
• Use Environment Variables where necessary

In short:

Things to leave out of VC:

• IDE settings (.project, .sln, .idea, ...)

Things to get into VC:

• Dependency Configuration
• Build Configuration (based on env variables / relative paths)
• Formatting preferences (optional)

Answer 2

This is a really broad subject so I'll answer at a very high level and not touch specifics unless they are relevant for OSS.

"Works for me"

This is really the crux of the problem; you have build environments that are different in a way that affects software behaviour. The industry at large has been tackling this for decades, and a number of best practices have emerged, such as making sure that development and production environments are exactly the same, or that all developers must set up their environments in the same way. Obviously some approaches work if you're paying someone full time to do whatever you want, but don't work if they are volunteers with their own devices.

Capture all configuration

Anything¹ that can affect your build - configuration files, environment variables, compiler flags, dependencies and versions - this all needs to be documented. Some go as far as recommending that these be version controlled, so environments can be replicated easily by checking out the relevant files.

¹_{except for secret keys and passwords.}

Continuous Integration

Write tests, run them with a CI service, for every build and every pull request. This should be your first line of defence - if it doesn't pass CI, don't check in.

There are many CI services available, many free for OSS.

Make sure everyone uses the same environment... but the onus is on you.

You can have the most complete walk-through on how to set up an environment but if it's going to take a day to complete, forget about volunteer contributions. Having a project be as easy to set up as possible is great, but almost a requirement for OSS. This might mean writing scripts that automatically installs dependencies, or sets up a testing environment, or having a suite of samples and templates.

GNU's projects are a good example of this in action; all their projects can be built by running ./configure; make. If you've ever had the misfortune of working with autotools, you'll really appreciate how much wizardry and blood/sweat/tears goes into making a project easily build and run on almost any developer machine.

Fortunately things are much easier these days; modern build systems, languages or frameworks do a lot of this work for you. But the onus is still on you to make sure new developers can easily get started.

Support as widely as possible

Which also means you often need to bend over backwards to support as many weird environments as possible, to minimise the barriers to entry. That is, instead of mandating The One True IDE, support all IDEs and editors. Don't use compiler-specific or bleeding-edge language features, stick to standard, widely-used ones. That also means keeping up to date with the latest developments, avoid using deprecated features, so your software doesn't break when built with the latest versions of its dependencies.

Answer 3

You don't say anything about what type of revision control system (RCS) you use. However, the type of problem you describe is exactly the type of problem Git, being a distributed RCS, was created to fix.

If everybody works on the master branch, or if everybody tries to consolidate every commit made by another team to their branch instantly, GIT will not help you. But Git will let you divide the project up in subtasks localized to specific areas of the code, and let a team create its own branch to work on its designated sub-task and not interfere with other areas of code. You then postpone merging a team's branch into the master branch until its sub-task is completed. This basically means that commits made by others will not affect you until you merge. When you merge, you consolidate everybody's work, and then everybody (for a moment) shares the code of the master branch, before continuing on their own branch.

Teams may also help each other between merges, but the general rule when you use GIT to coordinate work between distributed teams is that if somebody does some work on a subtask or code area assigned another team, they don't commit, but submit a pull request.

IMHO, if Git is used right it does a very good job of solving coordination problems. Git let each team work for a long time on a separate branch and check in their code in a shared repo without affecting the build of any other team.

When subtasks are complete, its branch need to be merged into the master branch. If the project is well managed and the teams disciplined, merging is painless. If they are not disciplined, Git will report conflicts which has to be solved manually. This may sometimes require some work, but is always doable.

There is no explicit mention of what RCS you use in the question. But since you talk about "pull" and "main branch" I assume you do, and maybe the one you use is GIT. If you do, and still have the problems described in the question, you need to look into how you use Git, and how you manage your distributed teams.