Strategies for Continuous Clinical Trial Management System (CTMS) Improvement

Strategies for Continuous Clinical Trial Management System (CTMS) Improvement


Hello everyone and thank you for your patience.
Welcome to the webinar entitled Strategies for Continuous CTMS Improvement, presented
by Param Singh, the Vice President of Clinical Trial Management Solutions. I am Eugene Sefanov,
the marketing manager at BioPharm. As a reminder, today’s webinar
is being recorded and will be posted on BioPharm’s website within 24 hours. I would now like
to turn the call over to Param Singh. Thank you Eugene. Good morning everyone. Welcome
to our webinar for strategies for continuous improvement, with CTMS. Let me start by introducing
myself. My name is Param Singh, I am the Vice President of Clinical Trial Management Systems
at BioPharm. I have been working in the industry since 1999 and have almost exclusively been
working with Siebel Clinical during that time. Before joining BioPharm I was part of Accenture.
Now I have had close to 17 implementations of Siebel Clinical under my belt and with
all of those implementations we also have created a Siebel Clinical accelerator, ASCEND,
and we have some other webinars related to that if you are interested. Now I would like to talk about my team here
at BioPharm. We provide a variety of services and products related to clinical trial management.
We manage implementations of Siebel Clinical, both custom and of ASCEND. We also manage
Oracle’s LabPas application, and incidentally, our next webinar in October is around the
LabPas application so if you are interested in that please register. Also if you know
our previous webinar you know you need to give plenty of implementations as well. Our
approach to implementations is very process-guided, we have services to help organizes design
and harmonize their SOPs and business practices with regards to Siebel Business management.
We offer comprehensive training services and products. This is a glimpse of some of the
services and products we offer, for more details, please contact me directly. Today’s Agenda. We are going to start out
by looking at the importance of a continuous improvement plan for your system, which will
consist of establishing a change control board to oversee review and approve enhancement
requests that go into the system to put a timeline to them and define the overall enhancement
process. We will then look at the sources of input to that process. Where we get those
enhancements, how are they being submitted, the sources of all of those enhancements and
enhancement requests
and also the review procedures of those enhancement requests. The frequency of those meetings
around the key resources, both from a business, technical resources, as well as your support
organization, how often they meet to review and gather the inputs to the continuous improvement
work stream. Then we will discuss how to manage those enhancements and make sure every request
is properly tracked and documented for criticality and effort. Then we will look at how to prioritize
those enhancements and allocate them for point releases. When it comes to prioritization
we are also going to go through a small example live today and look at ways to manage and
prioritize those enhancements. We will look at the timing of those enhancements as well
and how that will come into play in defining your continuous improvement process. And when
to make those enhancements to the system is going to be crucial in that and finally we
will cover some best practices for continuous improvement plans and have some time at the
end for questions and answers. If you do run into any issues during the webinar submit
your question or issues in the chat feature. If you have any questions submit those on
the chat function at any time during
the webinar. Let’s get started. I wanted to start by looking
at a pretty common scenario when it comes to implementing a system and it really doesn’t
matter when it comes to type of system that we are implementing but here is a pretty common
scenario. If you have implemented a system before you can probably relate to this. We’ve
just completed the initial release of the implemented solution, end users have just
gone through training and are ready to start doing their day to day activities, and in
the initial release, after all the requirement were captured, it is typical that not all
of your requirements made it into the first release. This is typical of any release where
based on budget or timeline or resources a subset were implemented. And many times these
phased approaches are preferable to the big bang approach, to reduce the overall change
impact on the users and reducing the time in delivering valuable functionality to the
users. Doing a phased approach is pretty advantageous but not all of the elements that were captured
were in the first release. It may only be used by a subset of users, and there is a
plan to roll it out to the other business users as well in subsequent phases. We also
have an annual budget meeting coming up where we can put in requests for budgets for other
planned projects. So this is a scenario where what do we do now, what is next? Having a
system implemented is only half the equation. It is important to develop a plan to determine
the long term success of that plan and high user adoption. A good way to manage post-implementation support
is to create a Continuous Improvement Plan. It helps to bridge the gap between traditional
help desk and support plan since it establishes a committee to oversee the post-production
process. When there is a formal plan and process in place, users will be less anxious about
being able to enhance the plan later and to include requirements in the long run. Knowing
that there is a plan in place for enhancements it frees the users from wanting to have all
of their requirements in that initial phase. Establishing the plan also encourages end
user communication and feedback. The plan also allows the system to be updated and flexible.
If a change in business process wants a change in functionality. It establishes an objective
way to evaluate potential enhancements and it is wise in that time and effort. It also
increases acceptance and usability because it calls for feedback. The
first step is to create a Change Control Board,
which will carry out the improvement plan. It should be made up of a mix of different
stakeholders. The business representatives provide the main information on enhancements
and usability of the system since they are the ones who interact with the system every
day, the majority of improvement input would come from them. The Clinical Administrators
would be able to put more input on questions and issues they have encountered from all
user support and daily operations. So it is important from either the clinical administrator
or help desk support to gather that information and help that has been requested by the users
so we can focus some continuous improvement enhancement on the things users have had the
most problems with. And then an IT project manager or someone from the IT department
should be there to help track the user requests, the level of difficulty of the proposed changes,
and also to help carry out those plans. Each of these groups and representatives are crucial
to having a well balanced, objective review board to handle requests and changes and make
decisions based on the organization’s needs. Next let’s look at how we define the enhancement
process. As part of the improvement plan you should define the following: How to collect
the enhancements, so what is the mechanism used to capture ongoing requirements, define
how often to review those enhancements, define how to prioritize those enhancements and when
to roll out those enhancements. And we will look at each of these in more detail. Now let’s look at sources of input for the
ongoing requirements. We already have our deferred requirement list from that initial
release. And from that initial release we’ve done our analysis and we had a series of requirements,
not all made it in, and some that were deferred make great enhancements. We also have direct
user feedback so after users use the system, they discover additional items which may be
helpful for their day to day activities. We have the help desk issue log which can indicate
which modules or areas users are struggling with, and we may want to focus some enhancements
around that functionality. Training session feedback, users typically gain a better understanding
of the system when they have hands on training and they will usually give feedback on that
system as a result during that training session. It is a good way to manage expectations and
give them the reassurance that their feedback is valuable and will go into this process.
The long term IT infrastructure roadmap, we need to look at it to determine potential
enhancements that may not have been available at the given time or which may become feasible
in the future. Integration with other systems and the desire to have that company wide integration
is going to be a key driver in some of the enhancements that you may make to your system.
User community conferences is where new ideas are presented, whether they are within your
organization or outside, that is a great way to get additional ideas of enhancements that
will ease key challenges. Each of these are great sources of input to your system and
you should be actively documenting these in a controlled manner to ensure nothing is being
lost. Next we have to define how often the resources
should meet to define newly documented requirements from each of the source inputs. Typically
what we have seen is the following. A monthly meeting with the end users to receive feedback
and then we have a quarterly meeting to review issues logged from help desk or training feedback
and a quarterly meeting for the change control board to review those potential enhancements.
This is just what we’ve seen in the past but ultimately each organization should define
the meeting schedule based on their own timelines, resources and availability. How do each of these groups meet to document
their enhancement requests? Most often, we have seen organization utilize a formatted
spreadsheet where they capture all the necessary information about that request. You can also
use some formality in a bug tracking system, you may use a support function to track those
bugs, but in any case you will need to categorize each request as a true enhancement or a defect
where something is just not working. You need to assess each request from different perspectives.
How critical, is not having it preventing them from doing their job, is there reasonable
workaround in the short term, how complex is the request from a design perspective,
does the benefit of the functionality and the enhancement outweigh the cost, how detailed
is the validation effort, does it touch multiple areas of the system which may require an additional
revalidation effort, does it impact multiple modules and pose any upgrade risks? All these
are things we need to assess about each request from those sources. How do we assess business operation criticalities?
At the very least, we need to identify the business requestor, capture clear details,
provide justification for the enhancement, what key benefits do they see in implementing
that enhancement, the impact on daily operations, potential alternatives or work around, capture
request date and priority ranking on that enhancement. This ensures that we are looking
at the business’
need for that enhancement. Determining level of effort for design and
development, we need to look at the actual effort involved in implementing that enhancement
and in this we need to look at the level of effort from a design and development perspective.
Just implementing a small change from a development standpoint could have a larger impact on the
design documentation as well as other documentation that go along with the system. Is custom coding
required? That could make it additionally complex. Can it be done in-house, or do we
have to get an outside vendor to get these updates, or can it be done with in-house resources,
and are those resources available? Does it build on the current system architecture?
Does it use the same platform and fit within that overall system architecture? Have we
identified different design and development approaches, so if we are looking at that assessment
or request, are there other ways to meet that key goal by doing something slightly different
in the system? Each of those things need to be looked at. We need to assess the validation effort. We
know quite well that even something that is quite small or minor can lead us down a path
of a significant validation effort. We need to look at the level of validation required,
the validation scenario complexity, is it a small change in multiple modules. We need
to look at how complex is this scenario, is it across multiple or isolated, and regression
testing, how much testing is needed based on the change, do we need to validate every
process downstream and upstream or is it pretty isolated? And on a similar topic, we need to look at
impacted modules and upgrade risk. What is the impact of the change on the existing functionality,
how many modules are impacted, what is the impact on existing integration that need to
be updated as well, and does the other system need updating based on this change in this
system. We need to look
also at any updates to the other system that we are integrating too. And finally, what
is the impact on future upgrades of the system? Is the custom coding put the software at risk
and are we willing to accept the risk? Is the benefit that it gives our users good enough
to accept a minimal amount of risk around upgrades? Next is how do we prioritize those requirements?
Now that we have captured all of the key things that we need to do around those enhancements,
how do we prioritize those enhancements? Is there adequate justification for the business
to make this a system change versus a process change? Can we identify a change in the process
that might achieve the ultimate goal. Ensure that all business units come together to evaluate
the requirements. Even if it is just one business unit that is submitting it it may have an
impact on other business units which have any sort of impact and whether they have any
feedback on how it gets implemented. You need to rank requirements by level of importance,
high, medium, low is one way but when users are submitting enhancements all of their would
be high so coming up with some sort of numerical or other objective ranking method works really
well and that we avoid everything being listed as high or medium and just focusing on leveling
of important of each of those enhancements as they relate to each other. When do we make these enhancements? Looking
back to our scenario we just went live with this system and are starting to record these
requirements. Our initial enhancements that we don’t make them until about three months
after the initial release. The reason is it provides the end users an opportunity to get
familiar with the system and in some cases requirements that were deferred may not be
requirements that they absolutely need in the system after they start using the system.
Requirements change as people start to use the system and understand how it meets their
business process. This gives our users time to refine those requirements and also during
this three month process we are also capturing support and help desk issues that our users
are having so it gives us some good feedback into potential areas that we need to address
sooner rather than later if we are seeing a lot of help desk calls around key modules
of the system. So that is where your initial enhancements come in, waiting that long to
put in some enhancements to a release system and subsequent releases or change controls
really depend on the budget and resources available. You may have minor releases 3-6
months or even a monthly enhancement and it might be minor and usually is but that is
what they have committed to the users. On a monthly basis the users can see additional
requirements rolled into the system. You do need additional resources to manage that but
minor releases 3-6 months work well and
you can roll the enhancements in with a major releases, so you can have a few minors then
a major release as a larger point release. With that we are going to look at a sample
exercise, a list of enhancement requests for a Siebel Clinical system and a spreadsheet
with a list of enhancements and we are going to fill out some of that information for those
requirements on that list. What we are looking at is an excel spreadsheet
that is used to track enhancements and defects that we’ve aimed to put into our system. What
we will do first is look at the fields we have available, everything we’ve just talked
about, request date, who the requestor is, the area of the application, the description
of the request, whether it is an enhancement or a defect, what the priority level is, right
now we just have low, medium, high and critical, a justification area for why that requirement
is critical or high, whether there is a work around or not, and fields to fill out around
effort and time and cost level, how many modules are at risk, and ultimately, our decision
whether we move forward and when it will be slated for. This is what we are going for.
A very simple spreadsheet. The first set of fields are what our users or our source input
will fill out, they enter in the enhancement and then we have an assessment piece that
we will do on the change control board side. Most of these are filled out but we are going
to go through an exercise and fill out a few of these. This first one is around protocols.
The ability to indicate that a subject has not filled out an informed consent and the
protocol does not need to complete an informed consent. This is an enhancement, the priority
level is low, the justification is we need the ability to track versions of the protocol
and subject visit template where a re-consent is not required or subject. Something might
have change din the version or the protocol was amended but the subject does not need
to re-consent. Is there a work around? Yeah, there are
comment fields and other fields we can use
comments on the protocol version itself, for us to comment on that, so there are workarounds,
they may not be ideal but they are there. Development and design time effort, minimal,
it is just an additional field, validation, minimal, modules at risk: protocol version
and potentially the subject visit template. In a future upgrade risk, there is no upgrade
risk. That is very simply how we would fill out this information. We need to be as objective
as possible, and if we are having the users fill this out we need to define what each
of these levels means, the priority level means. Low, medium, high and critical, if
they cannot do their job without the enhancement is critical. This is important so everyone
is comparing apples
to apples. Let’s go through the rest of these fairly
quickly. Add the ability to mark site and PR assessments N/A so they do not go into
the final score. The want to have some of the questions not apply to the overall score.
This is an enhancement and it is going to be a medium enhancement because there is some
configuration that we need to do, some questions are not applicable and they should not count
against the assessment score, no work around, moderate time, validation time moderate, modules
at risk and future upgrade risk. We won’t do these all in the interests of time. And
so as you go through this this might be something the users are doing or it is on the assessment
side, so as you go through this and have a priority level for all of them, a request
date for all of them, it is up to us to prioritize this list. You can prioritize it in a number
of ways. Maybe you have a directive to say we need to, if there is a defect we have to
do those first and then roll in any enhancements in that same change control. To do that you
may sort by the enhancement or defect first. Let’s look at that. First we might sort by enhancement or defect
and next we might want to sort by the request date so everything that has been requested
earlier that has been sitting in the queue we want to look at those, then priority level
as well, you might want to sort the priority level in some fashion. If you have instead
of high medium low you have a numerical order that is easier to sort. That way you can sort
this and look at the specific prioritization that your organization is looking at and the
great thing is you do not have to prioritize in the same way for each
change control. This gives you the flexibility
to prioritize this list based on whatever you organization’s needs are at the moment.
If there are things minimal in nature, they don’t have a large effort, then you might
want to prioritize those higher in the list because you can include them with very little
overhead if you are dong a change control anyway. Those are some examples of how you
can use and define a clear process of capturing these enhancements and defects and managing
and prioritizing them. We did it with excel but we have seen clients use bug tracking
tools like Bugzilla and others internally just to have a central place for everyone
to log their enhancements and defects. The sources can vary so if you want to give that
availability to everyone, the help desk and other folks to log these in a central area,
that might be something that you are going to want to look at. Okay, let’s go back to our presentation. In
summary, here is the continuous improvement process. We need to create the change control
board, define a continuous improvement plan, collect those enhancement lists from a variety
of sources, and once those are in a format for us to be able to look at and review and
assess and prioritize that list, once we determine the decision on which enhancements are going
into the release we need to determine the scope of that project and determine which
enhancements will be included in that release. We may have a budget for x number of days
for design and development so depending on what is in that list we may need to draw the
line and say we can only cover three or four enhancements during this change control. Define
the scope, prioritize resource control, determine the enhancements to be released and then we
will create a change control charter, open up the change control and all the documents
that go into that and kick off the project. Once you have kicked off the initial change
control the whole process starts again. You have the board, you have the continuous improvement
plan, now you need to run through those enhancements, determine the scope, each change control and
kicking off that process and you just need to repeat that process in an ongoing basis. Now we are going to go into some best practices
of a continuous improvement plan. Start thinking of your plan prior to the go-live. This should
incorporate deferred requirements, support organizations, key stakeholders and how to
create and improve requests in that system, you need to schedule recurring meetings to
establish the continuous improvement process and input key resources as well as for the
change improvement board itself to set those expectations. Afterwards, be sure to set things
up so users have a chance to use the systems and understand its functionality before planning
any additional resources. It is key to get users to use the system before unveiling a
point release. In the case of deferred requirements they may change in priority and additional
requirements may be found through the use of the system. On one final note the continuous
improvement process can also result in non system changes such as training manuals, business
process renewing, and additional documents. The focus should not simply be on changing
the system but an evolution of the process, technology, training and initiatives surrounding
it as well. With that we will move into your questions
and just submit those questions in the chat feature on your screen and we will address
as many as time allows. Eugene? ES: I will begin with a few questions that
we have already received. The first question has several parts. Who exactly should be on
the change control board, how many people, and representing which areas? Should one person
have the final say or it is always a majority vote. What do you recommend? PS: We have seen different things on change
control boards. Typically it is going to be somebody that has a budgetary responsibility
for the system, system ownership is one aspect, if the business is just submitting these requests
and the change control board is just assessing it is important to have some key business
leader on that board to help in that assessment as well. When we looked at business representatives,
IT project manager, and a help desk support people who are assessing and reviewing the
actual enhancements so the change control board can consist of a similar makeup to make
those final decisions. Should one person on the board have the final say? Depends on your
organizational culture. One thing that does need to happen is that you are looking at
these requests in a very objective manner and understanding what is being requested
and how that will change and benefit the user base. If there is an enhancement that is going
to reduce the level of effort in the system for a particular user role, it is important
to understand that key benefit and how it translates to overall business operations.
So as long as you have one person or a couple people making decisions and they understand
the decisions and the resources that are available, and can objectively do those assessments,
that is the key things. It should not matter if it is one person or multiple but as long
as they make objective decisions based on budget and business needs. ES: The next question is who typically maintains
the process log? Is it a project manager, a system admin, a head of change board, or
someone else? PS: That is a good question. We have seen
a variety of different things and in some cases we have seen the admins take a very
active role in this because from interaction with the users they have the most and so from
a support perspective, things that originate out of support questions, interaction with
the users, understanding their frustrations and being able to document those to the change
control process, we have seen that be pretty successful. If you have allocated a project
manager for the ongoing continuous improvement of the system and maintain that enhancement
log as well, and reach out to the key stakeholders. It needs to be somebody that has that interaction
with each of the folks that are going to be providing input and is going to be part of
this change control process, so the admins and the PMs are going to be part of that process. ES: What is the typical point release schedule
that you have seen in the industry? Twice per year, quarterly? PS: There is not a whole lot that is typical
in the industry. Every client has had a different scenario. I can tell you that a very aggressive
schedule is once per month they have committed to something, some new enhancement or change
to the system so the users know that if there is something they want to see, they have an
opportunity to raise that on a monthly basis and they do not have to wait a year to see
that change come through. And each company has different standards. We have seen companies
where each system has its own timeline for point release schedule and we have been at
a company where they have a set quarterly system schedule for all systems and all have
that same release schedule so you are bound by your company’s next system change. Your
CTMS or other system would fall within that schedule. That is again typically quarterly
or once every two months is pretty typical. ES: Where do patches from Oracle for the Siebel
fit in or is that handled separately? PS: Typically the patches from Oracle will
go into your continuous improvement work stream so it will be worked into the same way that
enhancements and defects are worked through. So it still will be logged and documented
and tracked and assessed in the same way. So when Oracle works through a patch release
for Siebel it may not be relevant for Siebel specifically because it is for the entire
Siebel and it may not have any for Siebel Clinical so you still need to do an assessment
of each patch and work it into a change control but you may choose to do Oracle patches separate
from any sort of enhancements that you are doing because then you can test the impact
of that patch on the existing system and then you can do your additional enhancements after
the fact. Working two at the same time may result in a little bit more of a validation
effort so you may want to separate the implementation from the rest of your enhancements. ES: We have one more question and I am trying
to have him elaborate. I think the question is can you customize a clinical trial management
system too much where it does not actually benefit the core product. PS: Is there a downside to over-customizing?
Absolutely. We have seen that in the past as well where we call it out as death by automation.
That was the key point we brought up, the change control process should not only be
looking at technical changes to the system but also at how the process may need to change,
addressing requirements through a balance of technology, process and training. Looking
at things coming through the help desk, if all users are having issues with an area of
the application, it may not be that the area of the application is problematic, it may
be that the training of that product may need to be enhanced up a little bit. What we have
seen through this same change control process, the deliverables may be work instructions
or quick guides or job aids to help the users in working with areas of the application.
I would encourage at least with the Siebel Clinical product, it is very configurable,
so it can be attractive to build codes to enforce a process but you have to address
requirements with a balance of process and technology. Don’t try to force the process,
you want the system to enable but not force the process. As you grow and scale through
acquisitions and mergers, processes are redefined. What you do not want is that every time a
process changes, that you have to change the system as well because you have made the system
so rigid it can only work with one process. So make the system flexible enough that you
can focus your enhancements on things that will improve usability but continue to make
the system flexible enough that you are not taking the human element out of the equation.
Don’t try to have your system dictate exactly how everything is going to run in the system.
That is the guidance that I will give and we have seen early adopters of the Siebel
system have seen that over-customizing is not advantageous to their long term vision
of clinical trials. Using that out of the box functionality or configuring the tool
with best practices that are established will ensure that you have a good system with good
functionality and low risk from an upgrade perspective. ES: Great, thank you so much. That will end
our Q&A session, if you have other questions we have some contact information in front
of you. Note that today’s webinar will be available on BioPharm.com within 24 hours
and I will send a link to the recording. We hope that the information we have provided
you today is helpful.


Leave a Reply

Your email address will not be published. Required fields are marked *