The expense in US patents doesn't come from the requirement that you prove you had the idea first. So making that the sole requirement would significantly reduce costs to file patent applications.
As it stands now, the disclosure requirement of US patent serves a different purpose -- to provide understanding of the idea to the public, a sufficient level of understanding that people of ordinary skill can make and use the invention. That's a lot of writing, and writing is hard. That's why it's expensive.
Off the top of my head, I'd say major components of expense come from two areas: (i) the best mode requirement and (ii) the harsh penalty for failing to disclose enough.
I'm not aware of any country other than the US having a "best mode" requirement -- a requirement that the inventor(s) disclosure any preferences in practicing the invention. Patents in other countries seem to be very brief relative to US patents, so maybe re-thinking the best mode requirement might lead to savings. Of course, it's there for a reason, but I have no problem having that reason reconsidered.
The penalty for disclosing too much is nothing, other than the superfluous effort in drafting the patent application. The penalty for disclosing too little is invalidity of any claim for which disclosure is inadequate. The result is that many practitioners don't try to cut it too close to the bare minimum required disclosure but rather over-disclose by a safe margin (at least one hopes). That added security isn't free, and many practitioners won't compromise that safety margin.
Perhaps a softening of the penalty would provide comfort to practitioners (or applicants) to get by with briefer applications -- a prorated reasonable royalty -- 90% disclosure gets 90% of a reasonable royalty.
That's just a wild, off-the-top-of-my-head guess at what we might do to make patents more affordable here. There are already all sorts of suggestions under consideration for improving the efficiency of the examination process, including separating out searching functions and heavy reliance on outside contractors in addition to the growing use of electronic information systems.
Regards.