Lets say a company spends effort in creating a language that addresses certain problems very
effectively. It if patents that language then any company wants to develop a tool based on that
language will have to pay some money to the original creators of the language.
Is my understanding correct?
Actually, the idea behind my question was that I am working on a rule based code generator
that automates certain steps in our domain. Once the language itself is defined, writing a
parser for that language is almost a no-brainer (more so with public domain compiler tools
like flex and bison) The algo that is used to parse the language cannot be patented since
most of the algo would probably come from bison itself. When I look at the solution, there
are mainly 2 parts to it:
1) rule based language that basically constrains the way algo will work
I thought this might be a new invention unrelated to what you were asking about before, since before, if I recall correctly, you were discussing rule-based code generation systems, and in this topic you asked about patenting a language itself.
Your rule-based system would still potentially be patentable, but would you really want to patent the tokens used in the source language? Consider it this way: if you did patent the tokens, all anyone would need to do to get around your patent would be to change the tokens. Instead of "if" they could use "whenever". Instead of "case" they could use "ifequal". Instead of "while" they could use "during". If you're asking about patenting syntax rather than tokens, consider that someone could simply swap a few tokens around in the grammar to work around your claims -- maybe use RPN to write expressions, or base it all on LISP (or, if you're doing that, they could base theirs on FORTRAN).
I believe that you will want to claim the inventive concept, not the implementation details. Mechanical device patents don't generally claim the diameter of the gears (at least not in the independent claims), they claim the way the various parts interrelate. Similarly, Sun has many useful patents on the implementation of their garbage collection system (note for non-software types: that's a standard technical name for the automated deallocation and reuse of previously dynamically-allocated memory which has fallen into disuse), on their bytecode interpreter, and so on.
The only time I've ever seen implementation-specific "language" details in the claims was with antivirus patents which were claiming certain basic search patterns for virus code. You might want to take a look at some of them to see how they go about making those claims, and then think about why they're claiming their invention in that way. It's not because their software itself requires those specific codes, it's because viruses (written for Microsoft Windows OS software running on Intel microprocessors, which is 100% of their target market) are constrained to include exactly those codes.
It's possible that a patent application for your code generation system will need to claim something that specific, but in my uninformed state I believe this to be unlikely. Please keep in mind that you haven't disclosed anything beyond broad generalities, so I'm writing this based on guessing what those generalities imply to me.
I'm reminded of a conversation I had with a friend, shortly after we both graduated from our respective CS programs. He was working for a small startup that was producing an OO language. I'd just finished digging through assembler code trying to find a C compiler bug (it wasn't keeping track of variable scope correctly when a local variable declared within a loop had the same name as a less-local variable). Anyway, he quipped that he wasn't even sure that programs written in his company's OO language could be written in assembler. I pointed out that their compiler/interpreter had to execute those programs in machine code eventually. While it would be a royal pain to write OO assembler, it's doable.
Similarly, while there's a possibility that your system and method could somehow require a certain language in order to function, it's nearly certain that it won't. You need to protect what makes your code-generation system unique, and it is much more likely to be something broader than the language itself. I find it hard to imagine a case where the language itself would be the uniquely useful feature. But maybe I am just not understanding what you mean by patenting the language.