Microsoft Word Packard Teaching Case revised docx

---

 

 

 

 

EVALUATION  TEACHING  CASE  

 

 

 

Evaluation  of  the    

David  and  Lucile  Packard  Foundation’s  

Preschool  for  California’s  Children  

Grantmaking  Program  

 

 

 

By  Susan  Parker  

Clear  Thinking  Communications  

 

 

 

September  2011  

 

 

 

 

 

 

 

---

Teaching  Case:  Evaluation  of  Preschool  for  California’s  Children

 

 

2  

Teaching  Evaluation  Using  the  Case  Method  

[From:  Patton,  M.  Q.  &  Patrizi,  P.  (2005).  Teaching  evaluation  using  the  case  method.  San  

Francisco,  CA:  Jossey  Bass.  pp.  5-­‐14.]  

 

“[Traditional]  evaluation  training…relies  mainly  on  traditional  didactic  teaching  in  the  

classroom  to  ground  students  in  the  scientific  approaches  that  are  the  cornerstone  of  the  field.  But  

methods  are  only  the  beginning  of  what  [students]  need  to  understand  in  order  to  succeed.  Once  

students  have  mastered  the  basics  of  evaluation  options,  designs,  and  methods,  the  challenge  of  

professional  practice  becomes  matching  actual  evaluation  design  and  processes  to  the  nature  of  the  

situation,  as  well  as  hearing  and  mediating  the  opposing  opinions  that  often  surface.  

 

In  mature  professions  like  law,  medicine,  and  business,  case  teaching  has  become  

fundamental  to  professional  development.  Once  one  has  learned  the  basic  knowledge  of  a  field,  

higher-­‐level  applications  require  judgment,  astute  situational  analysis,  critical  thinking,  and  often  

creativity.  Professional  practice  does  not  lend  itself  to  rules  and  formulas.  Decisions  are  seldom  

routine.  Each  new  client,  patient,  or  customer  presents  a  new  challenge.  How  does  one  teach  

professionals  to  do  situational  analysis  and  

exercise  astute  judgment?  The  answer  from  these  

established  professions  is  the  case  method.”  

“Cases  take  us  beyond  the  reality  of  the  individual  and  plunge  the  learner  into  a  plot  with  

multiple  perspectives,  strong  disagreements,  and  avid  articulation  of  fully  plausible  yet  fully  

divergent  views.  Just  as  in  real  life,  learners  hear  from  others  who  may  have  conflicting  opinions,  but  

unlike  reality,  learners  can  step  out  of  vested  interests,  remove  blinders  that  can  hinder  learning,  

and  experiment  with  new  skills  and  approaches  in  a  secure  environment.”

 

Best  teaching  case  practices  include:  

• The  core  decision  points  throughout  the  case  should  have  enough  tension  (and  enough  

factual  information  and  context  leading  up  to  them)  that  you  could  reasonably  argue  

competing  perspectives  about  the  decisions  made.    In  other  words,  the  case  should  not  just  

be  a  narrative  about  what  worked  or  did  not  work.  There  must  be  clear  moments  where  

decisions  could  have  gone  different  ways.  Choices  have  different  benefits  and  costs.  

• The  author's  voice  should  be  neutral,  with  no  "drawing  of  conclusions."  The  tension  between  

the  choices  at  the  decision  points  can,  for  example,  be  presented  through  direct  quotes  of  

the  participants.  The  case  itself  does  not  do  any  diagnosing  or  give  commentary  on  the  

success  or  failure  of  a  particular  decision,  nor  does  it  frame  or  summarize  the  questions  for  

discussion.  

• The  facilitator  should  be  able  to  ask  questions  like:    "What  is  the  main  tension  at  play  here?"  

 "What  do  you  think  about  the  way  the  group  decided  to  proceed?"  What  are  the  practical  

implications  of  the  decision  for  grantees?"  "What  did  they  give  up  by  going  that  route?"  

 "What  else  could  they  have  done  and  at  what  cost/to  what  benefit?"    

---

Teaching  Case:  Evaluation  of  Preschool  for  California’s  Children

 

 

3  

Background  on  the  Case  

This  teaching  case  was  written  for  evaluators  and  funders.  The  case  focuses  on  a  real-­‐world  

evaluation  supported  by  a  private  foundation  that  used  a  strategic  learning  approach  to  evaluation.  

Strategic  learning  means  using  evaluation  to  help  organizations  or  groups  learn  in  real-­‐time  and  

adapt  their  strategies  to  the  changing  circumstances  around  them.  It  means  integrating  evaluation  

and  evaluative  thinking  into  strategic  decision  making  and  bringing  timely  data  to  the  table  for  

reflection  and  use.  It  means  making  evaluation  a  part  of  the  intervention—embedding  it  so  that  it  

influences  the  process.  

Evaluation  focused  on  strategic  learning  is  different  from  more  traditional  evaluation  

approaches  in  some  important  ways.  For  example,  it  is  fundamentally  different  from  summative  

evaluation,  which  judges  the  overall  merit  or  worth  of  an  effort  for  the  purpose  of  concluding  

whether  that  effort  should  be  continued  or  discontinued.

1

 As  Michael  Patton  says,  summative  

evaluation  is  not  even  possible  with  emergent  strategies  because  they  will  not  “hold  still  long  

enough  for  summative  review.”

2

 Strategic  learning  is  also  different  from  formative  evaluation,  which  

focuses  on  improving  a  program  or  effort,  often  so  that  a  later  summative  evaluation  can  be  done.  

While  strategic  learning  certainly  aims  to  help  strategies  improve  or  move  in  a  positive  direction,  in  

reality  the  “right”  direction  is  not  always  known.  Strategic  learning  means  helping  strategies  adapt  

based  on  what  information  is  known  or  can  be  collected  at  the  time.  It  does  not  necessarily  mean  

making  judgments  that  what  was  done  before  was  ineffective.  Finally,  strategic  learning  is  different  

from  evaluation  focused  on  accountability,  which  aims  to  ensure  that  efforts  are  doing  what  they  

said  they  would  do  and  that  resources  are  being  managed  well.  Strategic  learning  has  a  much  

broader  purpose  that  goes  well  beyond  oversight  and  compliance.  

Specifically,  this  case  focuses  on  the  evaluation  of  the  David  and  Lucile  Packard  

Foundation’s  Preschool  for  California’s  Children  grantmaking  program.  It  is  intended  to  promote  a  

critical  analysis  of  the  evaluation  and  its  evolving  interaction  with  the  grantmaking  program  and  

strategy,  rather  than  an  analysis  of  the  grantmaking  strategy  itself.  The  case  chronicles  the  

evaluation’s  nine-­‐year  evolution,  and  identifies  key  points  at  which  it  switched  course  because  

methods  were  not  working,  or  because  the  Foundation’s  strategy  shifted.  It  highlights  several  

questions/challenges,  all  of  which  are  relevant  to  strategic  learning  approaches,  such  as:  

• How  to  ensure  the  evaluation  is  useful  to  multiple  audiences  (board,  funder,  grantees)  

• How  to  “embed”  the  evaluator  in  a  reasonable  way  while  maintaining  role  boundaries  

• How  to  manage  often  competing  learning  versus  accountability  needs  

• How  to  time  data  collection  so  it  is  “just  in  time”  but  also  reliable  and  credible  

• How  to  get  information  that  does  not  just  verify  what  program  officers  already  know.

                                                                                                                         

1

 Scriven,  M.  (1991).  Evaluation  thesaurus.  4

th

 edition.  Newbury  Park,  CA:  Sage.    

2

 Patton,  M.Q.  (2008).  Utilization-­‐focused  evaluation.  Thousand  Oaks,  CA:  Sage  Publications.  p.  118.  

---