Computer forensics, proprietary methods and peer review

As I prepare my course materials for the new course in Digital Investigations and the Law I find myself revisiting cases which I intended to blog when they were initially decided but which never made it to the screen. Here's an interesting one from 2005 which discusses when a court will compel computer forensics experts to reveal their proprietary methods, and which raises some interesting questions about whether such methods are compatible with the general approach of the courts towards expert witnesses.

In Mulcahy v Avoca Capital Holdings [2005] IEHC 136 (full text not available but summarised here) the plaintiff was the subject of disciplinary procedures by his employer including allegations of "improper dealing with the e-mail inboxes of senior members of staff and ... improper dealing with the company's IT systems". He brought an action in the High Court seeking to stop the disciplinary process.

In order to deal with the allegations against him, the plaintiff sought to have his computer forensics experts examine certain computers belonging to the employer. Access was granted by the court, but a dispute arose as to whether the plaintiff's experts would be entitled to keep secret their proprietary methods for carrying out the examination.

Significantly, Clarke J. held that while a court would not unnecessarily require an expert to reveal confidential methods, by acting as an expert witness a person exposed their methodology to scrutiny in court and fair procedures demanded that the other party be able to assess and challenge that approach in appropriate cases.

The relevant passage is worth quoting in full as the judgment doesn't seem to be freely available online:

The final point I would like to comment on is the argument put forward in evidence on behalf of Grant Thornton [acting for the plaintiff], which amounted to a plea for the protection of their proprietary methods. A court must always, in circumstances such as this, be concerned not to expose experts to any unnecessary exposure of the benefits of their craft, as it were, but it does have to be said that a person who presents themselves as willing to act as an expert in proceedings necessarily exposes their methods to investigation in court. Just to put it at its mildest, if Grant Thornton and Ritz [acting for the defendant]were to give evidence in a trial which conflicted as to their findings, the only way the court could resolve that conflict would be by investigating their methods and forming a view as to which method is better. So it seems to me, as a matter of principle and a matter of practice in this case, an expert just cannot stand on ceremony in that way; by being available to give forensic evidence in proceedings and expert is potentially exposing his methods to detailed investigation. He cannot say, "I am going to give evidence but I am not going to tell people how I carried out my inquiries." While a court should not make any directions that would unnecessarily expose the skills of an expert, it nonetheless seems to me that there is a limit to the extent to which those methods can be protected and, therefore, on the facts of this case I would not place any significant weight on that concern on their part. (Emphasis added.)

This decision is in one sense unsurprising: past decisions such as State (D&D) v. Groarke [1990] 1 IR 305 have shown a judicial willingness to look behind an expert's opinion to the procedure on which it is based.

But perhaps the most interesting aspect of this case, as compared with the use of other expert witnesses such as doctors or engineers, is the tacit assumption that computer forensics experts will be using methods which are confidential to them or home-grown.

Perhaps in the relatively early years of computer forensics as a discipline this assumption might have been justified - though today it's beginning to look increasingly shaky with the move towards open source forensics tools as well as commercial products such as EnCase. Nevertheless it raises an interesting question - should the courts accept expert testimony when the underlying tools or methods have not been the subject of peer review to ensure their reliability?

Although the Irish courts have yet to adopt an approach similar to the US Daubert standard, there has been at least one recent judgment in which "expert" testimony has been rejected where it hasn't been shown to have a "properly established scientific provenance" or "the requisite degree of expert peer approval". (See DPP v. Michael Joseph Kelly (2008) in relation to the controversial CUSUM technique for determining the author of a document.) In light of this decision, one wonders how the Irish courts might evaluate the use of proprietary computer forensics tools today.

For more on this issue, Meyers and Rogers (2004) is a good starting point.