Recently I've been working with some colleagues on a dataset. We've obtained annotations for several thousand images which we assumed to be licensed as CC-BY. We intended to release our dataset to the public also as CC-BY. Upon inspection of the metadata, some of the image licenses differ - e.g. there are CC BY-NC and CC BY-SA images hiding in there.

If we train a model on these images, assuming we take a conservative stance that the model is a derived work, the resulting model would be incompatible with one of the licenses (e.g. BY-NC and BY-SA). We can of course retrain our release models on permissive subsets of the data. We're also considering just throwing out the non-permissive images to avoid any legal headaches, but it's a waste of annotation cost and those data might still be useful to people.

The annotations are independent - that is, annotations for image A have no dependence on image B, for all pairs of images (A, B) in the dataset.

What is an appropriate way to release both dataset and models?

My thoughts are:

  • We can release the dataset in shards that correspond to the different licenses and have users pick which components they want. This leaves no ambiguity over what the licensing of each subset of imagery is. The non-permissive images are a relatively small subset (< 10%). There is a little more effort from users, but we can easily provide scripts/code to make life easier
  • Seems like we could put separate records on e.g. Zenodo and link them with a single DOI.
  • I assume that we should license the annotations under the same license as each image, as they're "derived works"? i.e. we can't just release annotations alone as CC-BY alone since to be remotely useful for end-users the images should be provided alongside.
  • Is a model that is trained on the full dataset for demonstration purposes able to be published at all? Ideally I would still like to report performance on the full dataset for academic purposes and because it should give the best results, but it's unclear to me if that model could actually be released. Or can we use a "research purposes only" argument?
  • From a publication perspective, I assume it would be totally legal to train only on the CC-BY imagery, run holdout tests on everything else and release the model as CC-BY as it has never seen the other data (this would be equivalent to publishing a model with no test-time metrics)

Whether a model trained on -NC or -SA data can be permissively licensed seems to still be an open legal issue and we would rather err on the side of the image copyright holders (and their licensing intent when they provided imagery), not what the image hosting service says.

To give an example of an application where this can be ambiguous: ImageNet is technically a non-commercial dataset but everyone pre-trains on it and a lot of people argue that the weights can be used in commercial settings. However, I'm aware that several large companies do not allow using it because of this uncertainty in licensing. A response to this is the PASS dataset which only contains images with known and permissive image licenses.

  • 1
    I am not convinced the annotations are a "derived work" of the images they describe. The usefulness of distributing the annotations on their own is not a consideration, but only how they came to be. If the annotations are formed by making changes to an image, then it would be a derived work. If they are created by a human interpreting the image and writing down the ideas/concepts shown in the image, then that does not create a derived work (because an idea/concept is not protected by copyright. Only the expression of the idea/concept is). Commented Jun 10 at 6:59
  • "but everyone pre-trains on it and a lot of people argue that the weights can be used in commercial settings" - OK, one could argue that, but didn't you just say earlier in your Q. that you want to take a conservative stance and assume that your trained model is a derivative work of the data you're training on?
    – Brandin
    Commented Jun 10 at 8:10
  • @BartvanIngenSchenau thanks - this is an interesting point that I've not found a solid answer for. This is an issue surrounding which image sources OpenStreetMap allow to be traced, for example (no Google Maps). In my particular example they are segmentation maps, e.g. traced regions - forming either a semantic mask or a list of poylgons in image coordinates.
    – Josh
    Commented Jun 10 at 13:14
  • @Brandin Yes, I give ImageNet as an example of contradictory practice in the field. Big companies that are sensitive to litigation don't take the risk, they use internal datasets that they have control over. I think the risk of litigation in our case is relatively small, but it's not worth the headache if there is a simple solution to definitively avoid it.
    – Josh
    Commented Jun 10 at 13:15
  • @Brandin for an example of a response to this issue, see the PASS dataset - robots.ox.ac.uk/~vgg/data/pass
    – Josh
    Commented Jun 10 at 13:21


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.