|
Hi, David, its been a while since I've heard from
you. Have you also changed employers? I'm now in pharma and using JMP in a
non-clinical environment.
Unfortunately I'm just starting to learn how the
validation stuff works. A lot of it seems totally illogical. I used
"validated software" when I was at BMS and it frequently gave me idiotic
results. Validation only checks for things that the programmers
anticipate. I'm much more comfortable with double-checking, looking at
residuals, asking whether the results make sense, and using the decades of
experience that I already have. But, like you, I'm going bite the bullet
and learn as I go.
The scripts that JMP generates give a semblance of
an audit trail, but I can imagine lots of ways for things to go wrong. For
example, the script doesn't tell one what data points were excluded or
hidden during an analysis. You could freeze every data table before
doing an analysis and the date stamps would
give one a complete but hard to understand audit. But what happens when
your Fit Model creates a column for predicted, confidence limit, etc? We'd
wind up with hundreds of data tables each of which was a modification of the
previous one. Following the audit trail would be a lot harder than just
re-doing the analysis.
I haven't the foggiest idea how one would audit the
installation. You run the install and see whether the software
works. I don't see how anything could go wrong other than the software not
working or the computer crashing, but I'm pretty sure our IT people do something
and then write a report.
I validated a front end for doing a Peto-Pike
analysis using SAS when I was at BMS. Everything that the programmer list
worked, but that's hardly surprising. However, in looking at some of the
analyses I noticed that all of the "missing rats" (it was just data
where we decided to imagine rats that went missing) had already been discovered
to have visible lesions. So we had data that was "missing not at
random". The "validation software" had all sorts of bells and whistles to
spot suspicious data, but it missed that one.
Are there readers of this list that have experience
with this sort of thing?
PS, I have a really bad cold so I apologize for the
lack of organization in the above note.
Emil M Friedman, PhD
emil.friedman@alum.mit.edu
18
Clifton Ave
Waterbury, CT 06710
216-287-0821 (cell)
MannKind
Corporation
Danbury, CT 06810
203-790-2507
2304 Richmond Road
Beachwood, OH 44122
----- Original Message -----
Sent: Thursday, May 15, 2008 11:43
AM
Subject: [jmp-l] Pharma/CRO JMP
Validation/Audit Trails
Has anyone in the
pharma/CRO industry gotten into validation of
JMP installation
and operation and getting an audit trail of script and data
table usage? A lot
of the industry does this routinely with SAS but I'd like
to talk with
anyone who has done something comparable with JMP.
Thanks,
David
..........................................
David N. Iklé,
Ph.D.
Statistical Science
Director
PPD Development
929 North Front
Street
Wilmington NC 28401-3331
Tel: 910-558-6678
Cell:
910-547-3564
Fax: 919-654-9348
______________________________________________________________________
This email transmission and any documents, files or previous email
messages attached to it may contain information that is confidential or
legally privileged. If you are not the intended recipient or a person
responsible for delivering this transmission to the intended recipient,
you are hereby notified that you must not read this transmission and
that any disclosure, copying, printing, distribution or use of this
transmission is strictly prohibited. If you have received this transmission
in error, please immediately notify the sender by telephone or return email
and delete the original transmission and its attachments without reading
or saving in any manner.
|
|