Research and Evaluation Practice Test

These questions are part of the practice quiz. Start practicing

• What are the four components of the CIPP framework?
• Even though there is a high rate of participants returning to homelessness, you decide to pull your sample from those that have done extraordinarily well.
• Coding a two-group treatment variable as 0 and 1 is an example of which type of variable?
• Which statement best describes the role of stakeholders in a data information system?
• Which is true about the literature regarding cost-benefit analysis of human services programs?
• ------ error occurs in a consistent pattern and biases outcomes, while ------ does not.
• Typical Case Sampling is used to select cases that represent an average or typical client.
• In a qualitative study, reaching a point where no new information emerges is called what?
• Assent in pediatric research is appropriate when?
• The program that provides the best results for the money is the most efficient. This statement is:
• What is the difference between random sampling error and sampling bias?
• What is the minimum grade a student must earn in Social Work Professionalism to be able to join the Social Work major?
• Providing a free online training program is an example of which category in program evaluation?
• Which statement best describes parallel forms reliability?
• Which of the following are common data imputation methods?
• The researchers surveyed those involved in an intervention designed to improve supervision. The supervisors in the sample represented what type of sampling?
• Which statement about p-values is correct?
• Which are common effect size metrics?
• Internal validity concerns the extent to which observed outcomes can be attributed to the intervention rather than other factors. Which two threats are common in quasi-experimental designs?
• A single-system research design may be used to evaluate an individual, group, organization, or community.
• Relative to other people your age, would you rate your overall mental health at this time as: Very poor; Poor; Fair; Good; or Excellent?
• If two instruments measuring self-esteem resulted in different scores for the same sample, this would indicate that the instrument lacked construct validity.
• When you finish analyzing interviews and share preliminary findings with some participants before finalizing the report, this practice is known as what?
• Which type of evaluation is typically used for program description, program monitoring, and/or quality assurance purposes?
• What is propensity score matching used for?
• In a simple linear regression model, what does the intercept represent?
• In program evaluation, volunteers, computers, students, and the Internet are typically classified under which resource category?
• Which sampling method is described as non-probability and purposive, often used with at-risk populations?
• Which statement correctly defines MCAR in missing data mechanisms?
• In an ___________ definition, you are assigning meaning to a concept in terms of activities or operations necessary to measure it.
• What is multicollinearity and why is it a problem in regression analysis?
• When evaluating a program, data should be kept in highly granular, individual records rather than aggregated to be most useful.
• ------ error does not bias outcomes
• Which two criteria are described as the core components of trustworthiness in qualitative research?
• Which of the following best describes a quasi-experimental design?
• In a program evaluation, researchers ask clinicians to name two or three 'average' clients to represent an average case. What sampling is this?
• Is the statement 'The least expensive program is automatically the most effective' true or false?
• Which statement about experimental designs is true?
• Which statement about IRBs is true?
• If numbers show suboptimal performance for an intervention, which type of evaluation helps identify the reasons behind this performance?
• Which description BEST matches maturation as a validity threat?
• Which statement distinguishes formative evaluation from process evaluation?
• What is the purpose of a data management plan (DMP)?
• What is parallel forms reliability and how is it assessed?
• In general, findings from single-system research designs cannot be generalized to a larger population because
• What term describes assigning meaning to a concept in terms of measurable operations?
• In a dataset where treatment groups A and B are coded as 0 and 1, the variable name and its values are which of the following?
• In which sampling method does the researcher decide the sample size, express it as a ratio of the population, and then sample every kth person?
• Which statement best describes a convergent parallel mixed-methods design?
• Which of the following is NOT a required section of an informed consent document?
• Explain confidence interval and its interpretation.
• Subjective data is often biased; the statement 'Subjective data should never be used to drive program decisions during evaluation' is
• Which statement best describes the difference between a systematic review and a meta-analysis?
• Which sampling approach is most likely to yield generalizable results when evaluating a program?
• Which term represents a socially determined code about what is right and wrong behavior?
• What is the purpose of a literature review in research design?
• In meta-analysis, what is a key quantitative outcome?
• Which statement best differentiates a theory of change from a logic model?
• The difficulty of assigning monetary value to some benefits derived from social services programs is illustrated by which example?
• Difficult-to-reach population scenario: you contact a few known individuals, ask them to participate, and ask them to recruit others, often with incentives.
• For a school-based program grant requiring both a treatment and a control group, what is a viable approach that does not deny clients access to needed services?
• Which design compares changes over time in the program area to changes in a similar area without the program?
• Define survey weighting and why it might be used.
• Rate your mental health on a scale of 1-10, with 1 being Very Negative and 10 being Very Positive.
• What diploma is awarded to graduates in this program?
• In simple linear regression, what does the slope represent?
• Why is it important to monitor client satisfaction and design studies to measure client satisfaction in social work?
• What is statistical power and why is it important in evaluation research?
• What is the primary function of open coding in qualitative analysis?
• Data from intake forms are most useful for program evaluation purposes when it is in aggregate form.
• Describe the difference between probability sampling and non-probability sampling, with an example of each.
• RE-AIM framework includes five dimensions: Reach, Effectiveness, Adoption, Implementation, and Maintenance. Which of the following is a RE-AIM dimension?
• Internal validity refers to which aspect of a study?
• If an evaluation shows all results are mixed or unclear, what is the recommended course of action?
• Which statement best describes the strength of causal inference in randomized experiments?
• Which is the first step in conducting a cost-effectiveness study?
• Modifying measurement instruments mid-study to improve data quality introduces which threat?
• What is a mixed-methods design? Name two common designs.
• Which example illustrates the challenge of valuing benefits in social services programs?
• In mixed-methods research, which data collection description correctly pairs qualitative and quantitative data collection methods?
• A typical focus of a process evaluation is to identify the degree to which the program is implemented as planned.
• How many group sessions did you attend in the past month?
• Which statement best describes the outcomes of qualitative data analysis compared with quantitative outcomes?
• Which statement best characterizes pre-experimental research designs?
• Which statement best describes a joint display in mixed methods analysis?
• Which design uses multiple baselines across participants, settings, or behaviors to demonstrate control?
• Which practice best aligns with ethical reporting when evaluation results show no improvement but could affect stakeholders?
• In a rigorous evaluation, the statement 'True' regarding data quality is correct if the assessment reflects factual accuracy of data collection and interpretation.
• Which term describes a design where participants are not randomly assigned to groups but receive an intervention?
• Causal statements in single-system research designs are based on what?
• In RE-AIM, which dimension focuses on the extent to which a program is adopted by target settings or staff?
• IRBs were mandated to convene for all of the following reasons EXCEPT:
• Cost-effectiveness studies can help administrators make decisions about which of the following?
• You attend one of the events hosted by the agency and ask attendees to participate in your evaluation.
• What is the role of outcome indicators in the first step of a cost-effectiveness study?
• When the response rate is low, you must be careful to avoid which of the following?
• Distinguish between formative and summative evaluation.
• How many youth will complete the training program according to the plan?
• Benchmarking serves what purpose in evaluation planning?
• In the X-O-R notation used in group research designs, what does the symbol R stand for?
• What is a joint display in mixed methods analysis?
• What is the recommended practice for defining success criteria in a new program?
• You ask the case manager of the agency to identify potential clients who would be good to interview. Each person you interview you ask if they know if anyone else who has participated in the program that might be interested in participating in your evaluation.
• What is p-value and what does it indicate in hypothesis testing?
• Which method provides evidence for convergent validity?
• What is grounded theory and how does it differ from content analysis?
• A clinical practice uses a waiting list control and compares outcomes with those who receive therapy immediately, without random assignment. What design is this?
• What evaluation design compares changes over time between an intervention group and a non-equivalent comparison group to estimate program effects?
• In qualitative research, using more than one type of data collection to corroborate findings and verify interpretations is called what?
• You are collecting data in a qualitative study through interviews with case managers. You begin to notice repetition in the interviews. By the tenth interview you are no longer finding new information. You have reached what is termed _______________________.
• In the described scenario, what term describes participants who do not complete the intervention?
• Which threat to internal validity occurs when natural development over time could influence outcomes in a pretest-posttest study?
• Which of the following is NOT one of the three primary purposes of process evaluation?
• An external evaluator is asked to report results showing no improvement for a program that could jeopardize staff jobs and agency reputation. What is the most ethical course of action before disseminating the final report?
• Explain the concept of construct validity and two methods to assess it.
• In cost-effectiveness studies, when the results show that a program costs the least to run, it should be considered the most effective for clients.
• When determining which treatment should be offered to a client, the choice should be the treatment that yields the most benefit while also __________.
• A logic model is best described as a framework that includes which elements and is used to plan, implement, and evaluate programs?
• Reliability refers to the
• The “Impact” (Long-Term Outcomes) section corresponds to which horizon?
• Which statistic describes the average distance of data values from the mean?
• ________ is a measure of how closely a variable's values cluster around a measure of central tendency.
• To be used in single system research, outcomes measures must be reliable, valid, and...
• Which describes the stated short-term outcome?
• When evaluating a newly developed program, which is the most appropriate time to define success criteria?
• Repeated measurements with the same instrument can lead to changes in results not caused by the program; this is called?
• What is thematic analysis in qualitative research?
• External validity refers to the generalizability of study findings to other settings, populations, or times.
• Which evaluation type focuses on how a program is implemented and the context, rather than final outcomes?
• Single system research designs are often criticized for their lack of generalizability, or application beyond the participants in the study. This is known as...
• Which of the following is NOT a component of a logic model?
• Findings from single-system designs are generally not generalizable to larger populations because
• Triangulation of data is best described as using multiple data collection methods to corroborate findings.
• Which term describes the tangible products produced by a program, such as a graduation diploma?
• Which statement best describes the role of a theory of change in evaluation?
• Which of the following best describes the short-term outcome described in the material?
• What type of design tests the same intervention with multiple clients all working on the same problem?
• What data type is used to categorize the minimum grade required to join the Social Work major?
• A single-system research design may be used to evaluate which levels?
• When planning a qualitative research study, which areas typically require the most work to meet budgets and timeline?
• Which statement correctly defines survey weighting?
• Which statement best describes a typical use of a logic model in program evaluation?
• Which item is NOT a required section of an informed consent document, according to the material?
• Define effect size and name two common metrics.
• A study designed to test knowledge after completing a course is an example of which research design?
• What is the primary purpose of multiple imputation in handling missing data?
• Which design collects data only after the intervention from a single group?
• What is intention-to-treat analysis in randomized controlled trials?
• Which statement correctly describes axial coding in qualitative analysis?
• How would you rate your overall mindset today, with options (1) Negative or (2) Positive?
• A cost-benefit analysis can help an administrator determine if the money spent on a program is related to its outcomes.
• Which two data collection methods are appropriate for process evaluation?
• Which of the following is a common data cleaning step before analysis?
• What is data triangulation in evaluation?
• An instrument can be valid but not reliable.
• External validity is influenced by the combination of factors including?
• In experimental designs, the control group primarily serves to
• In the described two-week intervention with high attrition, the term for participants who do not complete the study is Mortality.
• In qualitative data collection, which of the following are appropriate forms of data collection?
• Which statement best describes external validity with respect to sampling methods?
• Which statement about quasi-experimental designs is true?
• What is listed as an output of the program?
• What threat arises when comparison group is not comparable to the treatment group due to non-random assignment?
• A data information system ________.
• A program director wants to know if a permanent supportive housing program is less expensive than traditional emergency shelter services. What type of program evaluation would he or she conduct?
• Do you need IRB approval before proceeding with an evaluation in the given scenario?
• Which protective measure constitutes an 'additional protection' in consent processes for vulnerable participants?
• A researcher asks each worker to think about two or three youth that represents an ‘average’ youth in long term care based on demographics, length of placement, etc. What type of sampling is this?
• Looking for interview subjects who are extreme on a range of factors is known as what?
• Which of the following best describes the baseline phase?
• An evaluator observes classroom teachers to determine how well a new anti-bullying program is being implemented. This is an example of which evaluation type?
• What do you think about the requirement that all undergraduate students must take a class in Social Work Professionalism?
• Which sampling type cannot provide inferences back to the larger population, no matter how large the sample?
• Long-term clients who have been involved with a human services program may be prone to which bias when rating services?
• Which statement describes pre-experimental research designs?
• Which statement describes regression discontinuity design and when is it used?
• In research, the sample should ______ the population from which it is drawn as much as possible.
• If a program continues to perform poorly, what is the recommended action regarding continuation and replication?
• Which sampling method uses participants to identify other potential participants by referrals?
• In the X-O-R notation used in group research designs, which symbol represents observation?
• According to the ethical principle that requires researchers to maximize possible benefits and minimize possible harm, what is this principle called?
• What is the interpretation of a 95% confidence interval?
• A high standard deviation indicates a high degree of variability in a variable.
• In ITT analysis, what happens to participants who do not adhere to the assigned treatment?
• Which set of criteria are commonly cited to assess trustworthiness in qualitative research?
• Data visualization best practices for evaluation reporting.
• Interviewing child welfare workers on a break from a conference describes which sampling technique?
• Which of the following statistics should be reported for the variable of race?
• Single system research designs are considered qualitative research.
• In measurement, which statement best describes reliability and validity?
• Which of the following is/are an appropriate form of data collection in qualitative research?
• What are the three overarching purposes of process evaluation?
• What bias arises when the sampling frame fails to cover all elements of the population?
• What is an audit trail in qualitative research?
• Which set correctly identifies the four widely cited American Evaluation Association standards for research and evaluation?
• When would cost-effectiveness analysis be used?
• Which statement reflects the essence of grounded theory?
• A program is evaluated during a period of a major external event that could influence outcomes. This creates vulnerability to which threat to internal validity?
• Which statement describes sampling bias?
• Which sampling method relies on initial participants recruiting additional participants through referrals, often with incentives for recruitment, when studying a hard-to-reach population?
• Which evaluation type is primarily concerned with ongoing checks to ensure a program is delivered as intended over time?
• Which statement best defines user-centered design in evaluation?
• Which factor should guide the selection of an intervention to maximize effectiveness?
• Which term describes measures of progress toward a goal taken at intervals before the program’s completion?
• Minors (under 18) can give informed consent to participate in research studies.
• What is an IRB and what are common ethical requirements for research involving human subjects?
• What is the process called when researchers present findings back to participants to confirm accuracy?
• Which type of validity concerns the extent to which results apply to other contexts or populations?
• What is a sampling frame?
• Which type of evaluation focuses on whether a program is being implemented as intended?
• Which sampling method involves selecting cases that are unusual or exceptional to gain deeper insights?
• The baseline phase is used to establish a pattern of behavior before the intervention.
• Cronbach's alpha is best described as?
• Nonprobability sampling requires that each person in the population have a known and equal chance of being selected.
• In obtaining informed consent from vulnerable populations, which practice is NOT recommended?
• Which statement about bias in sampling is correct?
• For a nominal variable such as color category, which statistic is appropriate to report?
• Which evaluation is used to compare costs to outcomes without converting outcomes to monetary values?
• In single-system research design notation, A refers to the ------ phase and B refers to the ------ phase.
• In survey design, what is response bias and give two examples.
• Which sampling method involves using initial participants to recruit additional participants, often with incentives for recruitment?