As the creator of PACKTRACK and an expert in the field I’m commonly asked about extending our K9 record keeping system to compute an accuracy percentage statistic during K9 detection training.

The requested method of calculating this statistic is simple. If the dog finds or “hits” the target odor then it’s marked as correct. When the dog doesn’t find or “misses” the target odor then it’s marked as incorrect. If the dog indicates that a target odor is present when it isn’t (sometimes referred to as a “false alert”) then it’s also marked as incorrect. The statistic is calculated based on the percentage of correct responses.

The request for this feature may come from a misunderstanding of the 2013 Supreme Court case of Florida v Harris or be based on advice received from a trainer. The US Supreme Court ruling in this case explicitly rejected an earlier Florida court ruling that favored “rigid rules” in favor of a “Totality of Circumstances” approach. The key problem with using an accuracy percentage calculation in deployments is that it’s impossible to accurately say whether the dog was incorrect. The Supreme Court ruling went even further by stating that probable cause cannot be and has never been based on a “bright-line test” such as a statistical calculation. Probable cause is based on a “totality of the circumstances” analysis. Based on this decision, the idea of using an accuracy statistic for training would not be beneficial in court.

Another problem with this idea is it presumes that the outcome of a training exercise provides confirmation that the training was done correctly. It seems logical to assume that hits are correct responses while misses and false alerts are incorrect responses. However, this simplification reads more into the data than is captured. The idea that we can equate locating a target odor (a hit) to a determination of whether the training was done correctly is absurd. It’s like presuming that since no tangible substance was found after a K9 indication during a deployment that this indication must have been a false alert or somehow incorrect.

Behavior modification in drug detection training is mainly done with positive reinforcement at the appropriate time. Handlers can inadvertently train canines to respond to a “cue”, a behavior, sound, or action that the canine learns as a signal to respond in a positive way that a target odor is present. The canine learns this behavior over a series of exercises where the cue (an action taken by the handler) was done and the canine was rewarded in a positive manner. This cueing is consistently reinforced by the handler during training exercises and may cause what some call a “false alert”. While this “false alert” will be correctly marked as an incorrect response by the dog, it is actually a red flag that there is a training issue. The real incorrect training exercises are those prior to the “false alert”, when the handler performed the cue where target odor was concealed, and the canine was rewarded. During those exercises the canine was taught the cue. These exercises would be marked as “hits” with the presumption that the exercises were conducted correctly when in fact they were not.

An accuracy statistic requires that we read too much into the outcome of an exercise. There is no way to provide this statistic and we shouldn’t try. Training time should be focused on finding and correcting the problems that we, as handlers and trainers, may have caused. Training is the time to push teams to their limits and beyond. Handlers should come to training feeling free to make and learn from their mistakes so they can prevent and correct them in future training. Exercises should condition handlers to perform proper technique while increasing their knowledge and perfecting the skill of dog handling.

In using the proposed statistic handlers are forced to only work to the limit of correct performance. They will only do exercises that they know they can complete properly in order to maximize their accuracy percentage. Who would want to push themselves to the point of error if a statistic, which provides no explanation, will be used to gauge their reliability?

My advice is to be careful if you quantify your hits, misses and false alerts as a percentage of accuracy. You are keeping records that can negatively impact the quality of your training and provide a false sense of proper training. And, if you’re challenged in court, it’s unlikely that this statistic will do much to help you win a reliability argument. There are better ways to train and document your records and we will talk about this in future posts. Stay tuned.

‍

I periodically receive questions asking if the PACKTRACK record keeping system collects “double blind training” data. I was recently contacted by a new supervisor who was attending a class where this was discussed. It was his impression that the instructor was advocating for double blind training and that this was a necessary requirement to prove reliability. I can’t comment directly on the instruction as I wasn’t there. However, I do have considerable experience in this area. I’m also quite familiar with the views expressed by a range of defense experts, podcast authorities and trainers.

It’s easy to confuse the clinical definition of this topic with the practical application for police K9 training. In a double blind study, neither the participants nor the experimenters are aware of the test details. This helps eliminate bias and human influence. As a result, it’s considered the “gold standard” for scientific research. So why wouldn’t we use the same methodology for K9 training?

To properly explore this topic we need to start with a shared understanding of what “double blind” means in the context of police K9 training. Let’s see how it would work. In this type of training neither the handler nor the trainer can know:

•  If the target odor is present
•  Where the target odor is concealed
•  The intended purpose or goal of the training exercise

Think about what this might mean for the quality of training that you and your K9 receive. By applying this rigid model, the handler and trainer:

•  Don’t know if the final response was correct
•  Don’t know if the target odor was missed
•  Don’t know if the target odor was found using proper handling skills
•  Don’t know why a target odor wasn’t found (was it the dog, the handler, the environment?)
•  Can’t take corrective action at the appropriate time to effectively train and condition the behavior of the dog or the handler

This isn’t training, it’s shared ignorance. The timing of actions taken by the handler to correct or reinforce the behavior of the dog is critical to properly conditioning the canine. How are the handler, trainer and K9 supposed to improve under these conditions?

Rewarding or correcting a detection canine in an effective manner simply isn’t possible in a “double blind training” environment. Our goal is to train with intention and purpose. We want to correct or reinforce behaviors in a timely manner and push the handler and canine beyond their comfort zones. This is how we grow confidence and proficiency.

Please don’t misunderstand. I’m not against “double blind testing“. This is a very real activity that may be needed depending on the circumstances. However, “double blind training” just isn’t a thing. If you want to test yourself or your teams then please do so. Just don’t call it training.

‍