4

We are creating a Japanese Morphological Analyzer (https://github.com/ku-nlp/jumanpp), but it needs a segmentation dictionary to perform the analysis. We want to license the analyzer under a permissive license (we chose Apache 2.0 at the moment).

We use some Wikipedia and Wiktionary data for enriching the segmentation dictionary in addition to our internal entries. We want the overall dictionary to be permissively licensed as well, so it could be used commercially without problems and backward thoughts. What are our options for licensing the dictionary?

Improve this question
7
  • 1
    What do you mean by "permissively licensed"? Do you consider CC SA-BY to be permissively licensed?
    – Brandin
    Commented Dec 10, 2018 at 6:03
  • 1
    Strongly related: Using Wiktionary data in mobile app
    – Brandin
    Commented Dec 10, 2018 at 6:04
  • 3
    Could you say what you mean by "we use some Wikipedia and Wiktionary data for enriching the segmentation dictionary"? What and how you use the data will make a difference whether your dictionary is likely to be a derivative work or not.
    – Brandin
    Commented Dec 10, 2018 at 6:06
  • @Brandin We want the default dictionary to be usable in commercial (not only academic) settings. CC SA-BY allows that if I understand it correct. The question is whether the combined segmentation dictionary should be licensed under CC SA-BY.
    – eiennohito
    Commented Dec 11, 2018 at 3:45
  • We want to use only factual information from wiktionary entries.
    – eiennohito
    Commented Dec 11, 2018 at 4:02

1 Answer 1

Reset to default
2

Database rights (i.e. rights gained for creating and organizing a collection of data independently of the rights of each entry) and the legalities of data mining are highly jurisdiction-dependent. In some jurisdictions you may not need to acquire a license for source data used for text mining (e.g. under the fair use doctrine). Others make no particular exceptions, so you would be bound by the CC-BY-SA license.

The CC-BY-SA 3.0 does not have any provisions for text mining. That license allows you to create and reproduce adaptions as long as these adaptions are licensed under the same license, or a later version of the same license. I am not sure whether the definition of adaptions can include databases or the results from text mining. The CC-BY-SA 4.0 does consider Similar Rights incl. database rights, but you cannot directly use Wikipedia content under CC 4.0.

My interpretation:

  1. If your applicable jurisdictions allow the text mining you are in the clear. At the very least you want to avoid any infringement of copyrights and similar rights in your local jurisdiction, but you may also want to avoid infringement in jurisdictions where the data set will be published (effectively anywhere in the world). You may be able to license the database as a whole under whatever terms you choose, regardless of the licenses of entries within the database. Understanding how these laws interact may require professional legal help.

  2. If the applicable jurisdictions only allow data mining for non-commercial purposes, you may not be able to perform the mining as the open-source project might be used commercially.

  3. Absent any such permissions, you have to conform with the CC license you have received.

  4. The CC-BY-SA 3.0 license does not directly allow for text mining. But the intention of the license clearly is to allow any derivative uses. We can therefore consider databases created through text mining of CC-BY-SA 3.0 content similar to adaptions of CC-BY-SA 3.0 content.

  5. You may only publish these adaptions under a same or compatible license, in particular CC-BY-SA versions 3.0 or 4.0. Here, 4.0 may be preferable as it discusses database rights explicitly. Apache 2.0 or other permissive license are not compatible in this direction.

The Creative Commons wiki has a FAQ page about databases which may be worth reading.

Improve this answer
2
  • This section of FAQ page states that factual knowledge is not copyrightable. wiki.creativecommons.org/wiki/… If we are using only factual knowledge (e.g. writings, parts of speech and reading information) from the wiktionary entries, will that mean that we are free to do what we want with the data?
    – eiennohito
    Commented Dec 11, 2018 at 4:00
  • @eiennohito There are two distinct aspects you need to consider: (1) The process of text mining CC-BY-SA 3.0 covered works. (2) The copyright in the resulting database. You are right that knowledge is not copyrightable (which is why databases aren't exactly copyrightable either, but rely on Similar Rights). However, the process of text-mining necessary to produce this database is illegal unless you have a license or copyright exception to the creative works you are mining. My answer argues that the CC-BY-SA 3.0 could be giving limited permission as long as you satisfy the ShareAlike clause.
    – amon
    Commented Dec 12, 2018 at 13:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.