The CC-BY-NC-ND license allows you to
Share — copy and redistribute the material in any medium or format
That means, you can surely add it to your repository as long as the license stays intact. That means that your repository cannot be a pure open source repository anymore. Please make sure you associate the right parts of it with the right licenses.
If you ask Creative Commons how they define an adaptation they tell you it depends on the applicable copyright law (therefore depends on location). It may be hard to determine the status for code, since CC is not thought to be used with code. I would argue that the usage of the library should be allowed, so linking to it must be allowed too but there are still some problems. For example your own open source licenses code using this library might require also the library to be open source. However, you as the author, are not bound by your own licenses and can do whatever you wish, but others may not. It could become a mess.
The likely intention of the author of the library was to keep some control. However, that also makes it more difficult to use their work in meaningful contexts. I personally do not think that use of such a library constitutes a derivative work but rather constitutes use of the library, but you also have to check that the licenses of the other work in your repository are compatible with this. CC licenses should, in general, not be used with code. Just to be on the safe side I recommend to avoid using of CC licensed code if possible.