Health IT

hey all,

my name is yaw and i am a phd student in computer science at the university of washington. in places where there is no clean water or reliable power, one will often find mobile phones and internet access. my ph.d. work explores how to use the availability of that technology to improve the lives of the underserved. see more at http://cs.washington.edu/homes/yanokwa.

i primarily work on open data kit (odk), a free and open-source set of phone and cloud-based tools i co-created. odk has been used to create survey applications that embed images and location with survey data, clinical applications that provide video-enabled decision support for healthcare workers, and educational applications that assess student literacy. odk is being used by thousands of users all around the world and you can find out more at http://opendatakit.org.

i think most people on this list have likely heard about odk, and i'm glad to answer questions about the project, but i figured it might be more interesting to talk broadly about some of the challenges of open source.

gina trapani wrote thoughtfully on this a few months ago. in her piece at http ...

hey all,

my name is yaw and i am a phd student in computer science at the university of washington. in places where there is no clean water or reliable power, one will often find mobile phones and internet access. my ph.d. work explores how to use the availability of that technology to improve the lives of the underserved. see more at http://cs.washington.edu/homes/yanokwa.

i primarily work on open data kit (odk), a free and open-source set of phone and cloud-based tools i co-created. odk has been used to create survey applications that embed images and location with survey data, clinical applications that provide video-enabled decision support for healthcare workers, and educational applications that assess student literacy. odk is being used by thousands of users all around the world and you can find out more at http://opendatakit.org.

i think most people on this list have likely heard about odk, and i'm glad to answer questions about the project, but i figured it might be more interesting to talk broadly about some of the challenges of open source.

gina trapani wrote thoughtfully on this a few months ago. in her piece at http://smarterware.org/7819/my-codeconf-talk-your-community-is-your-best-feature, she noted, "I used to think that the process of making open source software went like this: you write software, you apply your open source license of choice to it, and you publish the source. Ta-da. Open source. But if you're building something you want people to use and developers to enhance, that’s only the beginning. At the heart of open source is one thing: public collaboration. Collaboration between humans who care about your software—your community. Your open source community will build, improve, tear down, rewrite, document, criticize, test, stretch, redefine, and give your software legs and a life that exists way beyond the original authors or any one person".

all that sounds great in theory, but what gina doesn't mention is how hard all of that is do in practice. on odk, we've had some reasonable luck, but it's been quite hard (read http://www.open-mobile.org/news/odk-and-javarosa-collaborating-code for a sampling). i'm curious what challenges this list has had with open source software. and in the spirit of open source, i'd love to hear how you helped that community overcome those challenges...

I and a Chilean clinician have started a company in Chile to implement open source clinical systems in Latin America, one of them being Open Data Kit Voice (ODK-Voice) and the other OpenMRS, and perhaps others to come. This is to say that we are a pretty experienced group in medical informatics, so take what I say with that point of view.
Having these two systems as a foundation has helped us to implement projects that woudl have otherwise taken us much longer to develop the software, not only from having the functionality that each (ODK-Voice and OpenMRS) have, but also that the communication between the two was developed for some of our requirements.
The other great benefit that we found was being able to build on top of those systems immediately, for example, we needed to have OpenMRS be able to schedule a call to a patient within OpenMRS and to do that we had to build it ourselves. Learning the code and having programmers who can do it have been challenges. I am familiar with the core group of programmers who are developing both of these technologies, but I think someone new, could have been intimidated by the ...

I and a Chilean clinician have started a company in Chile to implement open source clinical systems in Latin America, one of them being Open Data Kit Voice (ODK-Voice) and the other OpenMRS, and perhaps others to come. This is to say that we are a pretty experienced group in medical informatics, so take what I say with that point of view.
Having these two systems as a foundation has helped us to implement projects that woudl have otherwise taken us much longer to develop the software, not only from having the functionality that each (ODK-Voice and OpenMRS) have, but also that the communication between the two was developed for some of our requirements.
The other great benefit that we found was being able to build on top of those systems immediately, for example, we needed to have OpenMRS be able to schedule a call to a patient within OpenMRS and to do that we had to build it ourselves. Learning the code and having programmers who can do it have been challenges. I am familiar with the core group of programmers who are developing both of these technologies, but I think someone new, could have been intimidated by the amount of code and not have wanted to email the developer's list to find out how to do it.
However, the main obstacle that I'm finding is that there are a lot of myths about open source and what it means, that are usually negative. For example, I have come across people in the health system here in Chile that assumed that open source meant that it was developed by amateur programmers on their spare time, that there was no follow up to problems in the system, and that if they used any of these systems they would have to be on their own to maintain it. Dispelling those kinds of myths is more than a full time job for anyone, so even though we still use these systems, we have stopped saying that they are open source when we first talk to potential users. If they are interested we have no problem in talking to them about it, but we have found that if we bring it up, many people have a negative perception.
Another obstacle we have found, and this is not only for open source, is that it's a complicated and long process to translate what the user (such as a doctor or director of a project) wants into an actual system that they like or can use. This is helped by open source software in that you can show them a system that already works to bring up ideas, but is also hindered in that a structure has already been created, so if the user had another idea in mind that isn't compatible with the system it's much harder to change it.

Have you or others experienced this?

Joaquin

Hi Joaquin,

On Tue, 26 Jul 2011, GHDonline (Joaquin Blaya) wrote:

> Another obstacle we have found, and this is not only for open source, is
> that it's a complicated and long process to translate what the user
> (such as a doctor or director of a project) wants into an actual system
> that they like or can use. This is helped by open source software in
> that you can show them a system that already works to bring up ideas,
> but is also hindered in that a structure has already been created, so if
> the user had another idea in mind that isn't compatible with the system
> it's much harder to change it.
>
> Have you or others experienced this?

This happens all the time in software projects. It's one of the leading
causes of projects failing, especially big and expensive projects with a
large number of users, where it's seen as "impossible" to involve all of
them in its design and development.

A very common result is that a "system" is "delivered" at great cost,
which completely fails to meet the needs of its users but is imposed on
them by management, and it is therefore ...

Hi Joaquin,

On Tue, 26 Jul 2011, GHDonline (Joaquin Blaya) wrote:

> Another obstacle we have found, and this is not only for open source, is
> that it's a complicated and long process to translate what the user
> (such as a doctor or director of a project) wants into an actual system
> that they like or can use. This is helped by open source software in
> that you can show them a system that already works to bring up ideas,
> but is also hindered in that a structure has already been created, so if
> the user had another idea in mind that isn't compatible with the system
> it's much harder to change it.
>
> Have you or others experienced this?

This happens all the time in software projects. It's one of the leading
causes of projects failing, especially big and expensive projects with a
large number of users, where it's seen as "impossible" to involve all of
them in its design and development.

A very common result is that a "system" is "delivered" at great cost,
which completely fails to meet the needs of its users but is imposed on
them by management, and it is therefore hated, abused, ignored and usually
(mercifully) abandoned.

In my opinion and many others who practice Agile software development, the
only option is participatory inclusion of all users as stakeholders. At a
cost, it massively reduces the risk of project failure, improves the
quality and usability of the resulting package, and helps to win over
users, train them and get their buy-in to the project up front (during
development) rather than after development.

For example, see the Agile Manifesto, written by the flag-bearers of the
Agile movement, which states that they value:

* Individuals and interactions over processes and tools
* Working software over comprehensive documentation
* Customer collaboration over contract negotiation
* Responding to change over following a plan

<http://agilemanifesto.org/>

Cheers, Chris.
--
Aptivate | http://www.aptivate.org | Phone: +44 1223 760887
The Humanitarian Centre, Fenner's, Gresham Road, Cambridge CB1 2ES

Aptivate is a not-for-profit company registered in England and Wales
with company number 04980791.

joaquin,

i think the perceptions about what the quality of open source are
valid. there are plenty of oss projects where the tool just doesn't do
what it says on the tin. we certainly need to work harder as a
community to turn that around.

there are examples of this -- ubuntu and firefox come to mind as oss
projects where users are excited about using the tools. i think we can
also point to companies and tools that build on oss and are doing ok.
most of google's internal and external tech builds on open source, the
mac and the iphone wouldn't work without open source.

that said, it totally seems fine not to lead with oss. i think you can
save that when they ask about the licensing costs and how maintenance
and extensions can be carried out. that's where oss can really shine
-- it's about selling that value proposition.

as far as how to translate user ideas into code, that's an open
problem in software in general. i think what we do on odk works pretty
well. we don't necessarily listen to what our users say, we watch what
our users ...

joaquin,

i think the perceptions about what the quality of open source are
valid. there are plenty of oss projects where the tool just doesn't do
what it says on the tin. we certainly need to work harder as a
community to turn that around.

there are examples of this -- ubuntu and firefox come to mind as oss
projects where users are excited about using the tools. i think we can
also point to companies and tools that build on oss and are doing ok.
most of google's internal and external tech builds on open source, the
mac and the iphone wouldn't work without open source.

that said, it totally seems fine not to lead with oss. i think you can
save that when they ask about the licensing costs and how maintenance
and extensions can be carried out. that's where oss can really shine
-- it's about selling that value proposition.

as far as how to translate user ideas into code, that's an open
problem in software in general. i think what we do on odk works pretty
well. we don't necessarily listen to what our users say, we watch what
our users do and try to get to the core goal they are trying to
accomplish (given what where we think tech is going).

and if that goal isn't solved by a current tool, then yeah, you gotta
roll up your sleeves and start building. i do find that in most cases,
a lot of the pieces are there already and as programmers we have to
fight the urge to re-invent those pieces. that's where you need more
than programmers on a project to help guide those decisions...

as to agile (and xp and scrum and whatever), i must say i'm not
particularly swayed. when i was in undergrad, the hot thing was
waterfall methodology and uml diagrams. i put agile, or maybe "Agile"
in that same bin (trash bin. see rant at
http://steve-yegge.blogspot.com/2006/09/good-agile-bad-agile_27.html).

i think we need to be more up-front that software development is hard
and messy and failure-prone and more an art than a science (or maybe
more like gardening than construction). sure there are evolving best
practices we can follow, but i think success has a lot more to do with
team cohesion and vision than methodology.

moreover, i think any methodology (especially one that comes out of a
standard software company) is likely going to fail when it interacts
with the organizational challenges that most open source projects
face. agile pretty much requires face to face daily interaction and
that just doesn't work if your devs and users are in every timezone
and are volunteers and don't speak the same language.

this is why it's particularly important for everyone (especially
non-programmers) to participate. if you use an oss tool and it's
terrible, just helping organize regular developer and building user
guides can result in huge gains. when we walk together, we can walk
pretty far...

yaw

I found the following extract from Yaw's commentary very interesting as I am a "clinician end-user informatician."
I have seen over the years (since 1084) many of the applications and interfaces that work are trying to be re-invented. An excellent example is Summarisation. It is interesting that even though many in health say "I want my 'unique' data display format and in the end many have similar
features with minor variances.
>From my non-techo perspective having worked on a large MUMPS-based system and the early MMRS/AMPATH models open source systems seem to offer more adaptability to the users need and it is my impression that the sustainable "concept dictionaries" and form generators permit more
adaptability.
I hope this makes sense to the discussion group-remember I an a "doc" not and IT techo. Terry hannan

"and if that goal isn't solved by a current tool, then yeah, you gotta
roll up your sleeves and start building. i do find that in most cases, a
lot of the pieces are there already and as programmers we have to fight
the urge to re-invent those pieces. that's where you need more than
programmers on a project to help guide ...

I found the following extract from Yaw's commentary very interesting as I am a "clinician end-user informatician."
I have seen over the years (since 1084) many of the applications and interfaces that work are trying to be re-invented. An excellent example is Summarisation. It is interesting that even though many in health say "I want my 'unique' data display format and in the end many have similar
features with minor variances.
>From my non-techo perspective having worked on a large MUMPS-based system and the early MMRS/AMPATH models open source systems seem to offer more adaptability to the users need and it is my impression that the sustainable "concept dictionaries" and form generators permit more
adaptability.
I hope this makes sense to the discussion group-remember I an a "doc" not and IT techo. Terry hannan

"and if that goal isn't solved by a current tool, then yeah, you gotta
roll up your sleeves and start building. i do find that in most cases, a
lot of the pieces are there already and as programmers we have to fight
the urge to re-invent those pieces. that's where you need more than
programmers on a project to help guide those decisions...

Dr Terry J. Hannan MBBS;FRACP;FACHI;FACMI
Consultant Physician
Clinical Associate Professor School of Human Health Sciences,
University of Tasmania

Department of Medicine, Launceston General Hospital
Charles Street Launceston 7250

President Australasian College of Health Informatics(2007-9)

Visiting Professor, Universita di Modena, e reggio emelia, Italy
(Sept-Nov 2010)

Ph. 61 3 6348 7578
Mob. 0417 144 881
Fax 61 3 6348 7577
Email

Web/Blog: www.austemrs.com.au

I'm arriving a little late in the discussion, and a number of excellent points have been raised, so I'll do my best to address a handful of them.

My responses are a bit "meta", as they related to the platform we're all using to have this discussion. GHDonline has been built around Django (http://djangoproject.com), an open-source web application framework written in Python. The developers on the team have (with a few exceptions in the past) used Ubuntu as a development environment, and we've tried to connect with the local communities that have formed around Python and Django.

Agile: I used SCRUM (an agile methodology) for several years at a former job, and it worked well at both the small (6 devs) and large (100+ devs) scale. Previous job was waterfall, and it was often frustrating. However, I believe that the success/failure of each had less to do with the process itself than the _people_ and _implementation_ of the process.

People: Engaged product owners, who are willing to set priorities and say what is *off the table* in a release, is vital. These folks engage with the product's users regularly, in the manner ...

I'm arriving a little late in the discussion, and a number of excellent points have been raised, so I'll do my best to address a handful of them.

My responses are a bit "meta", as they related to the platform we're all using to have this discussion. GHDonline has been built around Django (http://djangoproject.com), an open-source web application framework written in Python. The developers on the team have (with a few exceptions in the past) used Ubuntu as a development environment, and we've tried to connect with the local communities that have formed around Python and Django.

Agile: I used SCRUM (an agile methodology) for several years at a former job, and it worked well at both the small (6 devs) and large (100+ devs) scale. Previous job was waterfall, and it was often frustrating. However, I believe that the success/failure of each had less to do with the process itself than the _people_ and _implementation_ of the process.

People: Engaged product owners, who are willing to set priorities and say what is *off the table* in a release, is vital. These folks engage with the product's users regularly, in the manner appropriate for the product. Yaw, your comment is dead-on: "that's where you need more than programmers on a project to help guide those decisions..."

> Yaw: "i think success has a lot more to do with team cohesion and vision than methodology"

Implementation: Couldn't agree more. On the GHDonline team, we use a modified SCRUM model, because we have a mix of technical (dev) and non-technical (community management) team members. The full process is too heavyweight for our needs. One of the keys I've found with any process is that you have to be willing and able to adapt it to fit your product, team, and organization. For us, Agile/SCRUM works best as a starting point for adaptation.

> Chris: "At a cost, it massively reduces the risk of project failure, improves the quality and usability of the resulting package..."

As one of the GHDonline product owners (Aaron VanDerlip will tell you that I spend far too little time coding these days), I've found this statement to be incredibly true. While I agree with Yaw that including "all users" as stakeholders is unrealistic, including a representative set of users as stakeholders in the process is well worth the cost.

Aaron

Joaquin brought up the issue of the negative reputation of OSS products, especially with regard to reliability and support.

If you've ever had to do long-term support for a piece of software you've built (or worse, for something someone else built), you know that this is an unglamorous and frustrating job that few are willing to do for free. Some rigor is required to keep the quality high. Toward this end, Yaw, I think the scales tip a bit toward the "science" end, unless you consider thorough documentation and rigorous unit testing in the "art" category.

The reputation of OSS is not unwarranted, but we are seeing an increasing number of OSS products that have strong communities supporting them. And that, backed up by the quotes from Gina, is where the success of long-term support for projects lies. Built a community around your OSS product that is passionate and committed, and it will live on even if you move on.

So, I have a question from my current point of view, which is a company that provides software for users that are not in my organization, so it's not as easy to sit with them or ask them constantly for their input. What would be your suggestions for this scenario?



Currently what we do is have meetings to define the initial requirements, and then have weekly meetings where we should the new developments for the projects. These meetings involve the user and at least one person from our team that can act as "translator" i.e. has both the vocabulary in health and IT to be able to speak both languages. However, it'd be great to formalize this process a bit more, so any hints as to other things to do or concrete ways to implement the methodologies you've talked about would be great. All of this knowing that these aren't the solution to all communication problems or delays in projects, but I think they would help a lot in setting expectations and keeping everyone (that is possible) updated on the project.



Thanks,



Joaquin



___________________________________________________________________

Chief Technology Officer, eHealth Systems Chile

Research Fellow, Harvard Medical School ...

I wanted to thank Yaw for his participation, as well as the members who participated in the discussion. For those who are interested in learning more about open source we've created a useful documents page on GHDonline here http://www.ghdonline.org/tech/discussion/free-and-open-source-software-foss-u...
(also anyone should feel free to add other information which might be useful).

If any one is interested in being on Member Spotlight, please email Sarah Arnquist at .

Yaw, thanks again for the thoughtful and very well referenced comments.

With this we close this Member Spotlight, though members should feel free to continue to comment.

Hello all,

Just my two cents: I'd like to reiterate Joaquin's points about the importance of 1) a scheduled meeting (weekly or otherwise) to keep everyone apprised of the project and 2) the presence of a "translator" in the meetings (and the whole project, really).

I drew these lessons from a simple database (for clinical and research purpose) project that went awry, because the programmers worked on it in a separate office and the clinical/research personnel didn't know the right questions to ask/didn't think to dig into specifics. E.g., by the time we found out that some fields were misnamed, they were already coded into the database so couldn't be corrected. There also might've been a bit too much trust from the manager that the lead programmer had the correct knowledge of data type and purpose (and not enough cross-checking of this assumption).

It's important for both users and developers to constantly cross-check their assumptions about what the desired application/system will do and how it goes about doing it. I.e., "there is no stupid question".


Baby M. Djojonegoro, MS MPH
Research Administrator/Epidemiologist
Curry International Tuberculosis Center, UCSF ...

Hello all,

Just my two cents: I'd like to reiterate Joaquin's points about the importance of 1) a scheduled meeting (weekly or otherwise) to keep everyone apprised of the project and 2) the presence of a "translator" in the meetings (and the whole project, really).

I drew these lessons from a simple database (for clinical and research purpose) project that went awry, because the programmers worked on it in a separate office and the clinical/research personnel didn't know the right questions to ask/didn't think to dig into specifics. E.g., by the time we found out that some fields were misnamed, they were already coded into the database so couldn't be corrected. There also might've been a bit too much trust from the manager that the lead programmer had the correct knowledge of data type and purpose (and not enough cross-checking of this assumption).

It's important for both users and developers to constantly cross-check their assumptions about what the desired application/system will do and how it goes about doing it. I.e., "there is no stupid question".


Baby M. Djojonegoro, MS MPH
Research Administrator/Epidemiologist
Curry International Tuberculosis Center, UCSF
3180 18th St. Suite 101
San Francisco, CA 94110
phone: 415-502-4886
fax: 415-502-4620
OR

www.currytbcenter.ucsf.edu

Hi all,

Another two cents on IT projects failing due to over-design and lack of
user involvement, this time from the UK NHS:

'A Public Accounts Committee report says mounting problems with the
electronic records system were making the £7bn [$10bn] project
"unworkable"...'

'He also said they were working on changing the proposed system "from a
programme that is top-down and centralised to one that genuinely is led"
by its users.'

<http://www.bbc.co.uk/news/health-14386753>

Cheers, Chris.
--
Aptivate | http://www.aptivate.org | Phone: +44 1223 760887
The Humanitarian Centre, Fenner's, Gresham Road, Cambridge CB1 2ES

Aptivate is a not-for-profit company registered in England and Wales
with company number 04980791.

Chris, I have taken the liberty of forwarding your comments to our College (ACHI) as I felt they are very relevant to our current discussions on the proposed Australian national PCEHR to be available for ALL Australians by July 2012!!!!!!! Terry

Sent from my iPad