Attorneys: do you know your allowance rate?

[mhgy]

I had recently someone ask me what my personal allowance rate was on prosecuting applications and when I said that I did not know it offhand, he thought I was an idiot--"what do you mean you don't know your allowance rate? It's your business!" I tried at one point to figure it out at one point because I was looking for objective ways to measure my own performance beyond the subjective approval/disapproval of my superiors, but there were so many conflating variables that I felt that the number became imaginary/pointless. I feel that the raw number of allowances divided by the raw number of applications that were assigned to me at the time of allowance does not really tell one anything about my performance. Do I count applications that I am pretty sure I could have gotten allowed but that the clients abandoned prosecuting for non-performance-related reasons (money, lack of interest in product, pursuing new application for new, more important embodiments, etc.)? Do I count appeals that have been pending forever that I am confident that I will win? How confident do I need to be to count these things? Do I just remove them from the numerator and the denominator? Etc.

Someone thinking I am an idiot does not bother me (or maybe it does more than I will admit it! Hah.), but I have been thinking about this a lot because, at least from what I can tell, there is not really a good objective way of measuring one's performance at "being a patent attorney." I think it is probably impossible to quantify what is a "good" argument and what is a "bad" argument without just analyzing the results, but results-oriented thinking is such a terrible way to evaluate your behavior in most endeavors that it frustrating to be forced to rely upon it. It's like giving 10:1 odds on a coin flip and then thinking you made a good decision if the coin flip comes up in your favor. Then again, the point of making the arguments that we make is to elicit a result in our audience (examiner or PTAB). If we fail to elicit the desired result, then maybe that is the exact objective result that we need to measure ourselves... but even then, I think an attorney can make what we (i.e. readers of this forum) would all agree would be a "good" argument (i.e. an apparently persuasively written argument grounded in fact, statute, and case law) and have it fall on deaf ears. But maybe we are just deluding ourselves and the only "truth" that exists in our profession is: "Did you get the allowance?"

What objective criteria do you use to measure your performance? Do you know your "personal allowance rate" offhand?

(Sorry, meandering musings while I chomp away on my lunch. Feel free to ignore. Hah.)

[ThomasPaine]

I have no idea what my "personal allowance rate" is.  People who think it's important are idiots.

[Robert K S]

"Personal allowance rate" is a fairly meaningless statistic in that it could have widely varying meanings depending on just how one practices and what philosophy of practice one espouses.  It's also a number that could be easily juked by filing only very narrow claims all the time in order to accrue easy allowances.  Personally, I would regard such work as unethical churning, since the resultant patents would be worth substantially less than well-prosecuted patents, and would, in most cases, be totally worthless in that they wouldn't cover anything that couldn't be designed around or superseded.  Since the name of the game is not just getting allowances but getting broad claims on commercially useful technologies that protect a client's business interests and create a valuable asset for the client, at best an allowance rate would only be one sub-metric, a component-metric, perhaps contributing to but by no means solely indicative of the overall effectiveness of a practitioner.

For a practitioner who appeals most every rejection, the "personal allowance rate" statistic might have a lag of 5-10 years behind the practitioner's current work, since that's how long it can take in some cases for an application to work through the appeals system to a satisfactory finality.  For some cases, that path only ends with a denial of cert. (or, much more rarely, Supreme Court affirmance of a rejection).  (When's the last time the Supreme Court took an appeal from the Federal Circuit on a prosecution issue?  I imagine it was one of the Section 101 cases, but I'm thinking that perhaps all the recent ones of those arose from litigation?)

As a wise attorney once told one of my attorney parents, if you're never losing cases, it means you're not pursuing tough cases that are worthwhile.  In the patent context, it also means, among other things, that one is seldom working to rescue those lost-cause cases poorly written or prosecuted by others but which, if allowed, would have value.

As an addendum, I can find one more reason that such a statistic, presented alone, could be found objectionable to one's moral palate.  A raw allowance rate might suggest that a practitioner's reputation, or charm, or inside relationship with the Patent Office, or these of the practitioner's firm, have more to do with winning allowances than technical understanding, legal acumen, persuasive writing ability, professional demeanor, efficiency, and the other valuable qualities that in a just world should be the ones that matter in winning allowances.

[JimIvey]

I have no idea what my "personal allowance rate" is.  People who think it's important are idiots.

I don't know if I'd go so far as to label such people as "idiots".  So few people understand what I really do that any question that seems like it could be relevant impresses me.  If I explain why the number is meaningless and they still think it's important, then yeah, they're idiots.

I use a question like that as an opportunity to education the (potential) client.  I think it's important that clients understand that patents (pending or issued) are valuable (and often expensive) property assets.  I can only tell a client how much an application is likely to cost and a very rough estimate of its chances of issuing as a patent.  That must be weighed against the value of the resulting patent and that's a purely business decision, not a legal one.  People who truly get that tend to be great clients.

Only certain types of failure to issue are under my control: 112, 101, and 102/103 in view of art of which I'm aware.  Aside from 101, I've never had any of those prevent an application from issuing.  101 is a work in progress, thanks to the ridiculous application of Alice by the PTO.  In fact, that might be a good answer to give someone asking about your allowance rate -- how many times something under your control prevented an application from issuing.

Things that are not under my control include art we didn't know was out there and the client just deciding that, in view of recent developments, the technology is just not worth as much as they thought it was.

Here's where I think I could do better.  Expense seems to grow exponentially as you escalate, first to the PTAB, then the Fed. Cir., then the S.Ct.  In view of the recent crap coming out of the PTO re Alice, it might be a "best practice" to explain that escalation through the PTAB and the Fed. Cir. is likely for a given case.  If a client could use that in determining whether a patent application is worth pursuing to avoid spending thousands if they know they'd never pay for it to go beyond the PTAB, that would be really good for them to know.  That's really hard to predict when an application is to be filed since appeal is usually years out and we rarely have comprehensive analysis of PTAB decisions like those given by Karen on an ongoing basis and by the Buckeye in a post in the last 24 hours or so (sorry for not having the Buckeye's real userid and post link handy).  Heck, if people want their patent practitioner to see into the minds of the PTAB years in advance, they should hire those big firm partners at $800 per hour or more.

I think the best way to answer a question like one about your allowance rate is to answer it honestly and to educate the questioner regarding what questions ought to be asked and about why good questions are good and why useless questions are useless.  Clients who appreciate and understand your answers are more likely to find synergy between their business and your services and stick around.  Others are more likely to move on to someone who fits their perception of a good IP practitioner and, in the long run, that's better for you.

Regards.

[lawstudent]

Since this question was not answered, here is the trivial calculation of allowance rate:

AR = #Issued / (#Issued + #Abandoned)

You can also determine an estimated allowance rate:

EAR = (#Issued + #Issuing)/(#Issued+#Issuing+#Abandoned+#Abandoning)

You could include in #Issuing the count of all application that have a Notice of Allowance, but are not yet issued. In #Abandoning, you could include all applications that are known to be in a client agreed state of abandonment.

[bartmans]

It is simple to calculate the allowance rate, but it is a meaningless statistic. It does not say anything about the skill of the attorney.
What if:
- my client decides to abandon the case because lack of commercial interest: one case less to be allowed, but not something I could be held responsible for
- my client decides to go for quickly allowable claims and decides to appeal on claims which are refused and later the appeal is dismissed? Does that count as an allowance or as a refusal?
- my client does not allow me to do a novelty search and thus lets me file an application for an invention that already exists?

The allowance rate is as meaningless a statistic as my hourly rate or the floor on which I practice.

[ThomasPaine]

"Since this question was not answered..."

It was answered.  It was clearly explained that it's a meaningless "statistic" and not worth discussing. 

Thanks for reviving this idiotic thread though.

[mhgy]

Thanks for reviving this idiotic thread though.

Well, sorry for starting an idiotic thread. They can't all be winners!

To be honest, the first paragraph ("What's your personal allowance rate?" story) was supposed to just be an introduction to what got me thinking about the actual topic I was curious about (objective criteria for measuring one's performance) because I agreed that it was a statistic largely devoid of reasoning and I did not want to be a slave to results-oriented thinking. Baseball is one of my favorite hobbies and I think a lot about the recent trend in the analysis of that sport to focus on process (sabermetrics) vs. results (conventional baseball statistics). It might be that our profession inherently lacks the ability for good process to be described objectively though since good process involves argument strategy (including writing style, logical organization, etc.) and interpersonal interactions, which inherently battle against objective measure.

... I suppose my writing was too rambling to adequately get my real questions across though. I blame it on apparently being partially distracted by eating lunch while writing.

I have actually been playing with tracking different argument types that I am using, categorized by certain criteria in the OA/references/app, and tracking the success each one has at obtaining an allowance. With a large enough sample, I would have an objective number for the % likelihood that each argument type will be successful with an examiner based upon the tracked criteria. Then one could determine what a "good" response would be by only utilizing an argument/arguments that have a high likelihood of success based upon the facts at hand. The challenging part is determining what factors are relevant and trying to create measures for certain things that are normally subjective that I would like to track.

For example an of a normally subjective criteria that would be useful to plug into the equation: In a 103 rejection, "how close" is the primary reference to the claims of the app being prosecuted? If the primary reference was very close, then the expected value might actually be highest to swing for the fences, so to speak, and utilize an argument that has a low likelihood of success, but would completely invalidate the combination of references if successful, such as a non analogous art argument. Conversely, if the primary reference is not close to the claims of the app (resulting in several references being chained together to create the 103), then it might be "better" (if we are defining "better" as "a higher likelihood for the response to the OA to result in a NOA") to go with a more conventional argument. How can we objectively define "how close" a primary reference in a 103 is to a claim though? Would it be the number of references in the 103 (i.e. if the 103 is based on a combination of 5 references, then the primary reference is not close)? Would it be the number of the claim elements disclosed by the primary reference (i.e. if the primary reference discloses 10/12 of the claim elements, then it is close, even if 4 more references are cobbled together to teach the remaining 2 claim elements)? Or, more likely, would one use several objective measures and just average them together to try and drown out the noise associated with each measure as with the now-defunct BCS computers (http://www.cbssports.com/collegefootball/writer/jerry-palm/24046194/bcs-computers-all-over-the-map)?

I think this all boils down to that I read too much about baseball and need less free time. Hah.

[khazzah]

Thanks for reviving this idiotic thread though.

Well, sorry for starting an idiotic thread. They can't all be winners!

To be honest, the first paragraph ("What's your personal allowance rate?" story) was supposed to just be an introduction to what got me thinking about the actual topic I was curious about (objective criteria for measuring one's performance) because I agreed that it was a statistic largely devoid of reasoning and I did not want to be a slave to results-oriented thinking.

... I suppose my writing was too rambling to adequately get my real questions across though.

The label "Idiotic" itself is certainly very subjective. I find the topic interesting, even though I didn't contribute.

Formulating a good question definitely takes time and effort, and most folks get better with time.

I see plenty of questions which I consider to be stupid, or at least uninformed, and I generally roll my eyes when I see them and then move on to the next topic. Fortunately, the regular contributors here all seem to have varied interests, so most questions elicit some sort of helpful response.

[ThomasPaine]

"The label 'Idiotic' itself is certainly very subjective."

But accurate.

[JimIvey]

It is simple to calculate the allowance rate, but it is a meaningless statistic. It does not say anything about the skill of the attorney.
What if:
- my client decides to abandon the case because lack of commercial interest: one case less to be allowed, but not something I could be held responsible for
- my client decides to go for quickly allowable claims and decides to appeal on claims which are refused and later the appeal is dismissed? Does that count as an allowance or as a refusal?
- my client does not allow me to do a novelty search and thus lets me file an application for an invention that already exists?

The allowance rate is as meaningless a statistic as my hourly rate or the floor on which I practice.

Those are all good examples.  I have a few more extreme ones.

I typically represent very young, emerging companies.  In 2001-03 and 2007-08, there was a huge flushing of small, young (under-capitalized) companies.  I've had as much as 2/3 of portfolios I was managing just go inactive -- mass abandonments. 

Between those catastrophic events, it was nearly impossible to get anything out of TC 3600, where much of my work goes.  So, for anything other than the most important patents, things often just got too expensive to keep fighting. 

None of those were my fault.  There are just so many more important things than abandonment to allowance ratio.

FWIW, I don't think I'm the only one with those experiences.  Nearly all the cited art I see in OAs is published applications and not issued patents.  I think a very large percentage of applications in my technologies never issued and never will.

Regards.