Re: [visionlist] [cvnet] Action needed on potentially damaging NIH Rule change


I think you’ve hit the nail on the head.  

Our discussion here reminds me, conversely, of a topic that raised a lot of discussion a couple of years ago regarding whether a claim that a commercial app, UltimEyes, that provided a training regimen involving visual search for low contrast Gabor patches, could be claimed to have been shown to improve vision.  The studies by Aaron Seitz cited to support that claim, of course, were not done using a random sample of the population, nor a placebo control group (what could that be?).  Thus, the FTC judged that the claims for the benefits of UltimEyes were fraudulent, and the Seitz was fined $150,000.  

Note that in that case, the careful psychophysical research by Seitz was deemed not to provide conclusive evidence of a clinical benefit to consumers because it did not fit the narrow definition of a clinical trial.  Many members of the vision science community wrote testimonials to the FTC to the effect that Seitz’s research was rigorous and showed, by normal standards in our scientific community, a real benefit of the training regime.  Thus, the implicit or explicit point made was that having a random sample of the population, or a double-blind placebo control experimental design was not a necessary condition to show a clinically relevant effect within the vision sciences.  Double-blind placebo control studies work well in drug effectiveness studies, but not necessarily for visual training regimen studies.

Here, the NIH seems to have expanded the definition of a clinical trial.  In doing so, it seems that such studies as those by Seitz COULD be labeled clinical trials, thus avoiding similarly punitive judgments by the FTC in the future.  However, in doing so, we are faced with the possibility that a broad range of studies without any real or potential clinical application might be labeled “clinical trials.”  This creates its own slew of potential problems, which have been described in earlier messages.

BTW, I am NOT claiming that the NIH changed its definition of clinical trials to solve the problems raised in the FTC’s UltimEyes case.  I have no idea whether there is any historical connection between the two–my guess is probably not.  

However, I think that both of these problems revolve around the definition of a “clinical trial,” and that the solution to both of them involves understanding that 1) a meaningful clinical application is at the heart of the study, and 2) the norms of the relevant scientific community as to what constitutes reasonable scientific evidence in support of such clinically relevant effects must be taken into account. 



On Mon, Jun 5, 2017 at 11:14 AM, Jonathan D Victor wrote:

I don’t think that a distinction based on timescale will work —  a clinical trial of a medicine to stop a seizure could be expected to have its effect within
seconds (for example), and a psych experiment might look at memory over months to years.


To me, the over-riding distinction is that of purpose:  in a clinical trial, the goal is to evaluate a treatment for a disease or condition, and a positive
or negative result of the trial will influence clinical practice – so there’s a rationale for having true clinical trials prospectively registered, to reduce the effects (on public health) of a publication bias towards positive results, etc., and to ensure
that anyone can find out about the studies and enroll.


In a typical vision lab study, whether our subjects are healthy or not, whether we randomize them or not, whether we are looking for effects that last seconds
or weeks, our purpose is to try to learn something about normal visual processing, and the results of the study may influence future research, but not patient care.


Meantime, *please write* to the NIH about the dangers of an over-reaching definition—




Jonathan D. Victor

Fred Plum Professor

Division of Systems Neurology and Neuroscience

Feil Family Brain and Mind Research Institute and Department of Neurology

Weill Cornell Medical College

v: 212 746 2343; f: 212 746 8050

lab home page:

personal home page:


From: [mailto:cvnet-bounces@lawton.]
On Behalf Of Sue LeatSent: Monday, June 05, 2017 11:47 AMTo: Susana Chung; Woods, RussellCc: McCourt, Mark; Peli, Eli;

Subject: RE: [cvnet] Action needed on potentially damaging NIH Rule change


To add my two cents worth :-


Firstly, I am wondering what problem they are trying to fix with this change. As it will cause a deal of additional problems and cost.


Secondly, it seems that the difference between a clinical trial (as we currently know it) and a laboratory study (e.g. doing a task in different levels of illumination,
but which would be randomised within subjects) is a question of the time. With a clinical trial (an interventions such as a pill or a different spectacle prescription, we are looking at an outcome in real life i.e over a period of days, weeks or months). But
for an experiment, we generally look at immediate results (within one day). Or am I missing something?


Thirdly, I would think that this would have a large adverse effect on education, in which students (even at the undergraduate level) do experiments, which would
now be clinical trials and become almost impossible in the time and with the resources that they have.




Susan J. Leat, PhD, FCOptom, FAAO


Adjunct Scientist, Lawson Health Research Institute

Head of Residencies,

School of Optometry and Vision Science (Room Opt 377)

University of Waterloo,

200, University Ave. West,

Waterloo, ON N2L 3G1, Canada

Phone 01-519-888-4567 ext 32040

Fax 01-519-725-0784


From: [mailto:cvnet-bounces@lawton.]
On Behalf Of Susana ChungSent: Monday, June 5, 2017 10:29 AMTo: Woods, Russell Cc: McCourt, Mark ; Peli, Eli ;
cvnet@lawton.ewind.comSubject: Re: [cvnet] Action needed on potentially damaging NIH Rule change


Hi All:


Just a couple of comments (more the consequences of the new definition):


1. Many journals (especially those commonly considered as “high impact” journals in vision science) now apply the “NIH clinical trial definition”, and will ask for a clinical trial registration number if subjects are from clinical populations.


2. Registering on is not fast.  It could take a month or so.




On Jun 4, 2017, at 6:36 PM, Woods, Russell wrote:


Hi All,
Sorry to disappoint on the within subjects design, as that is exactly the example given to the NIH committee by Cheri Wiggs (NEI program officer), who was told that it fit the definition of a clinical trial (the example that I mentioned earlier of each subject
having two luminance conditions).  Like you, I had hoped that this may be a way out.  Please note that the “new” definition places no limit on the number of study participants, so your “clinical trial” need only contain one subject.  Further, randomisation
of participants to groups or to interventions is not required, only that the intervention is prospective.
As noted by others, the only way out will be a ruling or guidances that give more clarity on how the rules should be applied and when registration on is required (at the moment, all are) and when GCP training is required (now, everyone must), so that we can go back to sensible conduct of clinical research.  As Steve Burns noted, such guidance would assist IRBs in making determinations.
As noted, make your opinions known to as many people at NIH as possible.
best wishes,

On 4 Jun, 2017, at 3:10 AM, Jonathan D Victor wrote:
If advocacy fails, this of course is a fall-back strategy – but I don’t think that it’s a good idea (pragmatically) to rely on a “loophole” of this sort.  It would be much better, from all points of view, for the NIH definition of a “clinical trial” to be sensible.
 Please write to the NIH with this in mind.
One could easily imagine a scenario in which an attempt to get around the new definition (by pointing out that one is using a within-subjects design) will  backfire – and that the study will be declared to be a “clinical trial with an inappropriate design”,
and therefore not even approvable by an IRB or a well-meaning oversight committee at NIH that thinks that it’s protecting human subjects from taking part in scientifically invalid research.  At my institution, I am routinely challenged on the justification
for my low number of subjects in psychophysics studies, since our IRB thinks in terms of drug trials.
From: [mailto:cvnet-bounces@lawton.] On Behalf Of Peli, Eli
Sent: Saturday, June 03, 2017 7:33 PM
To: McCourt, Mark;
Subject: RE: [cvnet] Action needed on potentially damaging NIH Rule change
Hi Mark,
As ridiculous as it may be I think you may actually be on to something valuable.
In fact we do not follow clinical trials rules in many of our studies and in particular in the within subjects design.
Furthermore when one runs a  legitimate clinical trial with a cross over design (to try to benefit from the within subject benefit), it considered unacceptable for the purpose of clinical trial design by the Cochrane type analyses.  For that they accept only
an analyses of comparison between the group first assigned to the first treatment and the group first assigned to the second treatment.

I have been objecting to this idea ,as it applies to many of our studies. Of course it is perfectly reasonable approach when you consider drug trials, which is what the whole structure of clinical trials built on and derived from.  In a drug trial cross over
(within subject design) may not be a reasonable approach in many cases.
In any case I think we should try to pursue this crack and see if indeed within subjects studies which will be inappropriate as clinical trials will be waved form this definition for the purpose of the new rules.  Since in many of our studies such study design
is preferred and universally applied – they can be declare non clinical trials.
Russell and Jeremey, could you pursue this idea with your respective contacts.?
From: [mailto:cvnet-bounces@lawton.] On Behalf Of McCourt, Mark
Sent: Friday, June 02, 2017 9:21 PM
Subject: RE: [cvnet] Action needed on potentially damaging NIH Rule change
1.     Clinical trials are a subset of clinical research. To be classified as a clinical trial, a clinical research project must involve the following three key characteristics:
·        Prospective assignment of subjects through a pre-defined process;
·        An intervention; and,
·        Evaluation of a health-related biomedical or behavioral outcome
Prospective assignment of subjects implies a between-subjects design. Are within-subjects designs exempt from these new definitions?
From: [mailto:cvnet-bounces@lawton.] On Behalf Of Jeff Mulligan
Sent: Friday, June 2, 2017 4:32 PM
Subject: Re: [cvnet] Action needed on potentially damaging NIH Rule change
On 6/2/17 1:26 PM, Woods, Russell wrote:
This is an issue that I have been working on with my NEI program officer, Cheri Wiggs, since December, and we have made no substantial progress.  Cheri is on a NIH committee that has been looking at the implications of the rule change, and though some guidance
on interpretation is expected, it seems that they are not backing down on the definition, despite the issues that you identified (and which I have raised with Cheri and she has discussed with the committee).
An example study that Cheri confirmed with the NIH committee is considered a clinical trial, now, is having a subject perform a task at two luminance levels (hits the two defining criteria: behavioural measure, intervention).  It’s crazy.
I found this document:  Questions and Answers Regarding the Revised NIH Definition of “Clinical Trial”
Here are some of the high points:
1. Why did NIH revise the NIH definition of “clinical trial”?
The definition was revised to make the distinction between clinical trials and clinical research studies clearer and to enhance the precision of the information NIH collects, tracks, and reports on clinical trials. It is not intended to expand the scope of
the category of clinical trials. (emphasis added by me)
1.     Clinical trials are a subset of clinical research. To be classified as a clinical trial, a clinical research project must involve the following three key characteristics:
·        Prospective assignment of subjects through a pre-defined process;
·        An intervention; and,
·        Evaluation of a health-related biomedical or behavioral outcome
I agree that considering two luminance levels to be an “intervention” is crazy.  It doesn’t seem to be that the problem is with the definition of a clinical trial, rather the interpretation.
Also, in many psychophysical experiments the outcome is not “health-related”.
This is all moot for me (government labs can’t get support from NIH).  But just when you think that the bureaucracy can’t get any worse, they surprise you…
my condolences,
Mass. Eye and Ear Confidentiality Notice: This e-mail and any files transmitted with it are confidential and are intended solely for the use of the individual(s) addressed in the message above. This communication may contain sensitive or confidential information.
If you are not an intended recipient, dissemination, forwarding, printing, or copying of this e-mail is strictly prohibited. If you believe you have received this e-mail in error and the email contains patient information, please contact the Mass. Eye and
Ear Compliance Line at 844-815-4401. If the e-mail was sent to you in error but does not contain patient information, please contact the sender and delete the e-mail.





Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s