As others have mentioned, if the European patent holder does not also have a U.S. patent (already issued, not merely an application), he's out of luck, at least until he gets a U.S. patent (assuming he has already filed an application which "claims benefit" to the European application). If he never gets a U.S. patent, then he has no case against you within the U.S., period. (If he currently only has an application, but eventually gets a patent, he may be able to sue you for reasonable royalties from the time his application was published until he gets an issued patent, but there are several conditions to that.)
You can search the U.S. patent application database to see whether the European patent has a corresponding U.S. application that claims benefit to it. A typical benefit statement is in U.S. 6788197 (the exact phrasing can vary, but it's always right at the beginning of the body of the patent or application). You might need to look for a U.S. application that claims benefit to the European patent or to the European application.
Read the "claims" section of the patent. The drawings don't really matter, what matters is what the patent claims cover. The drawings merely show one way to make the item. Read the claims broadly, and try to determine what each fragment of the claims mean (you will also have to read the description section to figure this out). If it says that the handle is connected to the lever, and if yours has a separate lever that is unconnected to the handle, then you're probably ok.
Assuming the worst case, that his patent claims apply to your item, and that he has (or gets) a U.S. patent:
Find out what the "filing date" was on the European application. Unfortunately for you, it usually takes between 3-5 years for a patent to be issued, so it is likely that his patent application was filed before you began selling your item. However, if he went through the "accelerated examination" process, he may have filed after you began selling your item. On the other hand, it might be an old product that you just began sourcing from the Chinese, long after they made it. If your item was designed before the European patent was filed, then your item obviously cannot infringe his patent. In fact, in that case, your item "anticipates" his patent application, and that makes his issued patent invalid.