Research Methods Resources

step 1 in a research project

 

Home

Research Methods Resources

Problem analysis

Quick links to some resources on this page

Tools for development from DFID (pdf 503 Kb)

Chapter 13 of Scientific Method for Ecological Research (Ford, E. David - 2000) (pdf 242 Kb)

Notes for internal reviewers (pdf 292 Kb)

Does your story "flow"?

 

We often see in drafts of student research projects conclusions that have little to do with the actual research project. For instance a research project has been carried out on in vitro propagation of Eucalyptus camaldulensis and the text ends with a conclusion "biotechnology reduces poverty in Africa".

Drawing unjustified conclusions from your data is just one of the reasons why a scientific paper is rejected for publication. To avoid such unsubstantial claims, it is necessary to start your research project with a thorough initial problem analysis. 

Many research projects go wrong right from the beginning even if the methods used are advanced simply because they tackle the wrong problem or loose the real underlying problem out of sight. The initial problem analysis helps you to keep your research focused and in perspective and to understand the logical relationships in the flowchart of the research process. If you get the connection between the problem within its context, the current understanding, study objectives and hypotheses flowing, the rest of the research will follow automatically to solid conclusions. The rest of the study is after all driven by the study objectives.

Sometimes authors of papers forget to place their research in context because it seems so obvious to them after working a long time on their research project. For readers of the paper the connections might still not seem obvious, so also here a detailed problem analysis is helpful for other people to help them put the research in context. 

Seeing the connections can be difficult. You have to think critically. And there exist no tools nor software that can do the thinking for you. Probably the best way to go is to discuss with supervisors, colleagues, experts and outsiders. Those discussions should be open and constructive, see more in the section on teaching approaches. 

 

Definitions, delineation and assumptions.

 

Ultimately everything is connected. The initial problem analysis is about cutting the problem into manageable portions and transforming them into researchable problems.

This process basically contains 3 steps:

  • Stating the problem.

Write a clear statement about the perceived problem. Revise to see if the statement is complete and grammatical correct. Check literature and discuss with others to see if you have a clear and correct perception of the problem.

  • Delineating the problem.

Cut the problem into manageable pieces that can be solved by a single research project:

  • Is there a clear, logical link between the development problem and your research problem ? 
  • Decide upon what is relevant and what is not.
  • Define hypotheses for each subproblem.
  • Give an explicit definition to each term you use. Be careful when using terms that have a "political career" such as "biodiversity", "poverty reduction", ... and be careful when using terms that have a different meaning in other disciplines. "Conservation measures" has for instance a completely different meaning for a soil scientist as compared to a biodiversity specialist.
  • It can help to write down what the study will NOT do.
  • Stating the assumptions.

State everything you take for granted and that has a connection with the research problem. Again, read outside your discipline to make sure that there is a broad agreement on your assumptions. For instance a statement such as "Planting trees on farms will reduce logging in natural forests and hence conserve biodiversity" might seem a logical deduction. However, some studies have shown that if farmers get more income, they might increase logging in natural forests by using the extra income to buy a motor chainsaw. Assumptions that hold after a critical review do not need to be investigated further.

Justify your research project by stating what benefits it will bring.

Finally, do the "So what ?" test. Ask yourself the question if your research is really useful in solving the development or livelihood problem. Imagine your research project has been carried out and ask yourself "So what ?".

Tools for problem analyses.

As mentioned above, critical thinking and a good discussion with others is probably the best approach to the initial problem analysis.

But when dealing with complex problems, you will still get stuck somewhere. In such a case, using facilitating techniques to structure the discussions and using elements of decision science can help.

The links on the right give details on Decision Protocol 2.0. It is intended to be a tool to help US Forest Service decision teams work through complex business and environmental decisions. It is an administrative aide that introduces the professional to the principles of decision science, outlines useful steps, and provides sources of information and techniques for improving decision quality. You find more general information in the Introduction.

 Introduction 

 Process cycle

 Problem cycle

 Design cycle

 Consequences cycle

 Action cycle

 Appendix A: Tips and tools

 Appendix B: Comparisons with DP1

 Appendix C: Further reading

 Appendix D: References

 Appendix E: Summary of core questions

 List of summary tables

 Glossary

While this tool is essentially a tool for managers, you could use elements of it to help structuring the initial problem analysis of your research project. Look especially at the questions 6 to 13 of the Problem cycle and adapt them to the context of your research. Appendix A gives tips for the facilitator of the discussion and also explains more on tools to use.

Decision Protocol 2.0 was copied with permission from the CRAFT website (Comparative Risk Assessment Framework and Tools). The source URLs is:

http://www.fs.fed.us/psw/topics/fire_science/craft/craft/Resources/Protocol/index.htm

Reference to the Decision Protocol 2.0 should be cited as: Berg, J., B. Bradshaw, J. Carbone, C. Chojnacky, S. Conroy, D. Cleaves, R. Solomon, S. Yonts-Shepard. 1999. Decision Protocol 2.0. USDA Forest Service, Ecosystem Management Coordination Staff, FS-634.

Another tool that can be easily adapted to analyse a research problem is the problem and situational analysis often used when designing development projects. Basically you organise a brainstorm session in which participants write perceived problems (existing negative situations) on cards, you tape them on a wall and identify the focal problem, arrange the cards in such a way that they form a problem tree showing cause and effect and inverse the problem tree into an objective tree showing means and end. Each means-end branch is now a possible solution to a part of the problem. You use all kinds of criteria to select the most feasible one. When using this approach to analyse a research problem, the most feasible means-end branch will be a researchable problem that is most likely to lead to solving an existing livelihood or environmental problem.

This picture shows an example of the initial phase of a problem and situational analysis when developing a project proposal on the conservation and restoration of natural forests in the border area of DRC, Rwanda and Burundi. Cards (perceived problems - in French) from participants to the discussion were taped on a wall and are being rearranged to build a problem tree.

initial phase of problem and situational analysis

 

Click on the link right to see the finished problem tree of the above mentioned project proposal. The software used to draw this problem tree is SmartDraw. Another possible software is Microsoft Visio.

problem tree conservation restoration.pdf

 

The problem tree was inverted and rewritten as a goal tree.

 

goal tree conservation restoration.pdf

This is of course an example of an analysis of a huge and complex problem. No research project could possibly solve the main problem. But as a researcher you can now position yourself in the bigger picture and contribute to solving part of the problem.

Imagine you're a botanist dealing with medicinal plants. In the problem tree you see that overexploitation of certain plant species, among them medicinal plants, cause part of the loss of biodiversity and that the local population prefer medicinal herbs instead of medicine from pharmacies because of lack of medicines or the fact that they are too expensive. One of the means in the goal tree has become "to produce medical herbs outside the forest" so that medicinal plants in the forest will be harvested at a more sustainable level. 

As a botanist specialised in medicinal plans you have now a new problem, but this time it lies completely within your field of expertise. A medical researcher could of course look at the same causal relations and means-ends links from another angle, for instance how can we produce medicines that people need in a cheap and affordable way? But those are still not researchable problems. 

So the botanist has to do another problem analysis, this time more narrow and focused. Why don't people grow medicinal plants outside the forest ? Is it because of availability of seeds, problems with seed germination, difficulties when growing the crop, lack of knowledge, difficulties in processing the harvested plants, ....? Which species do they need, what quantity, ...? Invert this problem tree to a new goal tree and following one or more of the branches will have brought you one step closer to defining a manageable research problem. Perhaps it will even be necessary to do a third problem analysis, this time for instance on the problems encountered when growing one particular medicinal species and build a new goal tree.

Eventually you will have identified a researchable problem and you will have sound and logic arguments why this specific research contributes to solving a major environmental and livelihood problem.

More on the problem and situational analysis can be found in chapter 3 of the DFID Tools for Development handbook. It also contains chapters on stakeholder analysis, risk management, participatory methods and facilitating, among others: toolsfordevelopment.pdf

The "Tools for development" handbook was copied with permission from the website of DFID (UK Department for International Development). The source URL is:

http://www.dfid.gov.uk/pubs/files/toolsfordevelopment.pdf 

There exist many more advanced tools that can be of use. In the following PowerPoint presentation, Dr. Keith Shepherd gives an overview of some soft systems tools: value-focused thinking (with some agroforestry examples), systems thinking and soft systems methodology.

Problem analysis- Keith Shepherd.ppt (17 Mb !)

The context

 

As a researcher, you don't work in isolation of what is happening in the world. 

This presentation by Dr. Frank Place gives an overview of the link between research and development projects, impact pathways and about different stakeholders who might hold different subjective assumptions about research needs.

Research and development - Frank Place.ppt

Also as a student you should relate your research project to the national, regional or global context. In this presentation, Dr. Tony Simons points out that students everywhere in the world face similar challenges. He gives a list of 10 policies and programmes you should be aware of if you are working in the broad field of natural resources management.

Your research in perspective - Tony Simons.ppt

Creativity and critical thinking

 

Research should draw conclusions and inferences beyond the point already achieved within the existing body of knowledge, as is reflected in the existing literature and should be able to make some kind of prediction instead of just describing a given situation at a given location. Two important characteristics of any researcher are creativity and critical thinking.

In this presentation, Dr. Paul Woomer gives some elements of creativity and also puts research projects in a broader perspective: who are the clients and what are the benefits of a research project.

Innovation and problem solving Woomer 2004.pdf

On the following webpage, Prof. Jere Lipps has summarized the skills involved in critical thinking and rules for evidential reasoning together with simple techniques to achieve them in two tables.

This is science.pdf

The next contribution warns about the impact of the mass media when informing about scientific research and how researchers should be actively involved in correcting the media if they report wrong on scientific issues. It is written from an US perspective, but the phenomenon exists worldwide.

Science in the mass media.pdf

"This is science" and "Beyond reason: science in the mass media" were copied with permission from the website of Professor Jere H. Lipps of the Department of Integrative Biology and Museum of Paleontology, University of California. The source URLs are: http://www.ucmp.berkeley.edu/people/jlipps/science.html  and http://www.ucmp.berkeley.edu/people/jlipps/Beyond.html 

 

Recommended literature

Ford, E. David (2000) Scientific Method for Ecological Research. Cambridge University Press. 564 pp. ISBN 0 521 66973 1 

Click here for the online catalogue page of the book at the Cambridge University Press website. Click here for a pdf of the table of contents.

Review from the publisher:

Scientists tend to take the thought processes that drive their research for granted, often learning them indirectly by observing their supervisors and colleagues. This book emphasizes the advantages of being explicit about these thought processes and aims to help those undertaking ecological research to develop a critical attitude to approaching a scientific problem and constructing a procedure for assessment. The outcome is a text that provides a framework for understanding methodological issues and which assists with the effective definition and planning of ecological research. As such, it represents a unique resource for anyone embarking on their research career. It also provides a valuable source of information for those more experienced researchers who are seeking to strengthen the methodology underlying their studies or who have an interest in the analysis of research methods in ecology.

David Ford is a Professor in the College of Forest Resources at the University of Washington, USA. His teaching spans courses such as Scientific Method, Ecological Modeling, Spatial Processes in Ecology, and Ecology of Managed Forests and Ecosystems.

This book covers what most other books on research methods do not cover: the thinking, philosophy, reasoning, strategies and social interactions behind scientific research. The topic is not easy, but the author illustrates the progression from vague ideas into research plans and theories through examples from real research projects. The first section of the book is about developing an analytical framework: conceptual and propositional analysis for defining research problems, developing a research plan, theories, principles of measurement, methods of reasoning, assessing postulates and philosophies behind different steps in scientific research. While the examples in the first section come from ecological research, the whole section is of use to researchers working in other disciplines, especially the parts on when to assume something and when to investigate and the differences between axioms, postulates, hypothesis and theory. 

The second section, "Making a synthesis for scientific inference", takes you to a more abstract level and covers topics such as properties and domains of ecological concepts, advancement of scientific research programs and the use of mathematical modeling in ecology.

Scientific research has the connotation of being intrinsically objective, but in reality it is very much a social process. Scientists are supposed to live according to several norms but often act according to counternorms. Fraud and misconduct does happen and gender does play a role. The third section, "Working in the research community" explores the sociology of science and also discusses the creation and use of scientific literature and the influence of values and standpoints on conducting and funding scientific research. The third section, just like the first one, will be of value to researchers in many disciplines, not only ecologists.

We are very grateful to the author and Cambridge University Press for permission to put a copy of chapter 13 "Scientific research as a social process" on this website. Click here to read the pdf version.

The fourth section, "Defining a methodology for ecological research", discusses progressive synthesis and criticism of ecological research and suggests ways for improving ecological research.

Generally, Scientific Method for Ecological Research is not a book that will read in one go. It is a book to read and reread in parts and pieces, each time comparing what you read in this book with your own experiences and what you read from others.

Teaching

 

As mentioned above, a good and well-structured discussion with supervisors, colleagues, experts and outsiders is the best way to help a researcher or student work through the initial problem analysis of his or her research project.

A possible teaching approach is to organise a scientific writing workshop. Not the kind of scientific writing workshop that focuses on how to write a scientific paper (style, structure, etc.), this can come in a later phase. In this type of scientific writing workshop however, the training is organised along the different steps of the scientific method as described above. Participants are asked to present a PowerPoint presentation (alternatively use a flip over chart) to the group consisting of 6 slides (one for each topic and allowance of one additional slide to present the evidence):

  • What is the development problem?

  • What is the current understanding?

  • What are your study objectives and hypotheses?

  • Present the evidence using 1 graph or 1 table.

  • Give your interpretation and does it solve the problem or does it updates the understanding. This has several advantages.

First of all, by presenting their research project even at an early stage, it helps participants focus on the main points and makes them think about any inconsistency between the different steps and helps them define the problem and study objectives more explicit. Second, it makes them aware of the importance of sharing and discussing their work with their colleagues. The culture of discussing your work and showing your data to colleagues seems to be lacking in many research organisations. After the presentations, resource persons and participants give feedback.

Scientific growth depends on scientific criticism, the same statement applies to the growth of individual researchers.

"At any time, science, that body of knowledge that describes our universe, is imperfect. The descriptions it offers are the best we have, but they are incomplete, imprecise, awkward, and limited. Sometimes, they are wrong (Weisskopf, 1984). The recognition of these shortcomings is essential to the growth of science, because they show us where improvement is necessary." (...) "Progress in science demands that the future overwhelm the past, correcting its misconceptions, improving its descriptions, and discarding its mistakes. Criticism lays the ground work for scientific advance."

Peters, Robert Henry (1991) A critique for ecology. Cambridge University Press, Cambridge, UK. 366 pp. ISBN 0 521 39588 7 - p. 14 - 16

Weiskopf, V.F. (1984) The frontiers and limits of science. Daedalus, Journal of the American Academy of Arts and Sciences, 113, p. 177-195

It is of course much easier to criticise other people's work than to write a good research project yourself. It is important to realise that scientifc criticism is not so much about criticism on itself but about effective and constructive criticism. Adhering to a set of rules for internal reviewers will avoid excesses. This means that the participants have to be:

  • Open. They have to say what they think.

  • Complete. They have to cover all the important points.

  • Constructive. Instead of just criticising what is wrong, they have to suggest ways to improve the research. The reviewer should be a collaborator looking for ways to improve the paper and not an inspector looking for mistakes.

More specifically, they should be looking to see:

  • Whether the research addresses relevant issues or whether there is a need for authors to enlarge/diminish their outlook.

  • Whether authors have overlooked an important issue.

  • Whether the implications and conclusions drawn by the authors are rigorous and warranted.

  • How these implications fit with other research being done on related topics. 

This framework can be easily adapted depending on the available time and the level of skills and experience of the participants. If the level of the participants is rather low, it can be incorporated in a one-week data management course that focuses on data management using Excel. The first days, the resource person can give some short lectures and exercises, followed by work on their data. Then ask the participants to give their presentation and present the evidence using a table with means or a graph. Based on the problems that surface during the presentation, the resource person can add some additional lecture modules, demonstrations and exercises. In most presentations there will be inconsistency between the different steps, the problem and study objectives will be vaguely stated, the evidence will not answer the hypothesis, graphs and tables will be difficult to interpret and the interpretation and conclusion will not be based on the evidence provided by the data. This means the resource person can plan ahead and have a set of short PowerPoint presentations, exercises and additional background material at hand. Then the aim of a next workshop is not to just look at the means but take the variability into account. This means introducing some statistical topics and applying them immediately to the data of the participants. If the group is more experienced, there will be less need for lectures, demonstrations and organised exercises. Then the aim of the workshop becomes to take away any obstacle that blocks progress on writing the paper. The role of the resource person is then to ensure participants use their time efficiently, to review drafts and to encourage progress, with some technical input from time to time. The resource person can randomly distribute draft papers of each participant to two other participants one day before the PowerPoint presentations so they have time to review. This approach will be difficult for a resource person who doesn’t know the type of research the participants are doing. However, anyone working in a biometrics or research support unit has enough knowledge of the research agenda of his or her institute and can even estimate the level of skills of each individual participant.

If participants don't have data yet but have an idea of which research they want to do, ask them to present a table or graph with expected results.

This approach can also be adapted to teaching undergraduates at a university. Give a first assignment to each student to write a presentation on the research he or she plans to undertake. Next, divide the class in groups of about 6 persons and give each group a second assignment to write a review of minimum 3 pages on 1 or more randomly selected research projects. Adapt depending on class size and available time. 

Following link gives an example on the rules set out for conducting an internal review at ICRAF: internal review

All this means that the role of the resource person or lecturer for this type of exercise is that of a facilitator. You can find some tips and tools for facilitators in Annex 1 and 2 of the above mentioned toolsfordevelopment.pdf and in appendix A of the above mentioned Decision Protocol 2.0.

 

Home

Research Methods Resources

   

GenStat Discovery Edition