The problem is that the RAW implies only things considered creatures caught in the area take damage.
That would also mean Fireball only does damage to creatures, and everything else is just ignited and only if they're flammable? Worst game ever.
Edit: Wait a minute. Player Handbook, Chapter 8
Characters can also damage objects with their weapons and spells. Objects are immune to poison and psychic damage, but otherwise they can be affected by physical and magical attacks much like creatures can.
Am I missing something here? Why isn't Prismatic Wall affected? Are walls not objects?
DMG, page 246 mentions walls specifically:
Use common sense when determining a character's success at damaging an object. Can a fighter cut through a section of a stone wall with a sword? No, the sword is likely to break before the wall does.
Common sense, my worst nemesis 😔