Both of these rooms have the ability to delight a few and intimidate many. If you aren't into plants and don't read much, then having a library or sunroom in your home is a little like growing an extra toe on your left foot. It's inconvenient and pretty useless.
You may adore activities that would make either of these rooms a wonderful indulgence, but when you move, don't expect most potential buyers to share your passions. To a degree, the same can be said for built-in bookshelves and those little, bump out greenhouse windows, too.