It sounds like you're putting good thought into this.
The two main concerns seem to be whether or not the goods truly compete and whether or not the other company is actually making a "trademark" use of the name.
In thinking about trademarks, it helps to keep in mind their function: to help consumers distinguish among goods/services in the market place so they can rely on reasonably consistent quality from a given brand.
The hallmark of infringement is "likelihood of confusion", which is determined by the similarity of the marks (how they sound and look) and the similarity of the goods (including where they're likely to be sold and to what sorts of consumers). If the marks in question are identical then you mainly focus on the goods. How likely are people to think a certain name brand on X means the X's come from the same source as the company that puts that brand on Y? Delta faucets and Delta Airlines? they can co-exist without confusion. Joseph Banks socks and Joseph Banks hats? probably not. (I say "mainly" focus on the goods because the "strength" or distinctiveness of the mark at issue can play a role.)
So depending on how distinctive your mark is and how analogous your example is to the actual situation (and you're wise not to use the real facts in a public forum), there could well be a conflict.
Regarding the other company's rights, as you've noted they've put the world on notice of their claim to trademark rights by using the "TM." So you know they won't take kindly to any sort of competitor using the purported mark, and that notice helps them show willful infringement if they have the rights they claim. In the US, trademark rights come from use of the mark; registration only enhances those rights. If they are using the mark in connection with sales of the product, I believe that can be sufficient without their actually applying the mark to the product or packaging. (Active practitioners, correct me if I'm wrong here.) So, depending on the mark (e.g. the generic term for a product can never be a mark for it) they could have enforceable rights.
The analysis of trademark conflict involves a mixed bag of competing influences and is best left to someone with experience in how courts have been parsing similar situations. If you have the resources to do so, it would be money well spent to hire a trademark attorney to check on the viability of not just this, but any other mark you might consider prior to use or attempt at registration. (And expect the results of the analysis to be in terms of how likely the mark is to be clear of problems, there are never any guarantees, only strong probabilities.) If that's out of the budget then it may be best to err on the side of caution and follow the maxim "when in doubt, don't."