I found the answer by mousing over the icons in the "existing treaty" part of the diplomatic interface.
If a treaty can be cancelled, you do it by left clicking on the icon. You can check by looking at the "treaties sent" part of the diplomatic interface.