1 of 9
15.S03
Experimental Innovation Lab
MIT SLOAN SCHOOL OF MANAGEMENT
Spring 2017
Tuesdays 2.30-5.30pm
Syllabus
Instructors:
Don Sull (
dsull@mit.edu
)
Neil Thompson (
neil_t@mit.edu
)
Teaching Assistants:
Caroline Fry (
c_fry@mit.edu
)
I. INTRODUCTION
Evidence-based decision making is hard. Managers want to learn how to achieve their goals by
mobilizing their resources and capabilities, but doing this requires evaluating the outcomes of their
projects and initiatives, and knowing how to interpret them. The gold standard for building such
knowledge is randomized experiments. From science to medical trials, this is the tool that is used when
practitioners really want to know what causes something.
In recent years, businesses have started to harness this powerful tool to help them achieve key business
goals. Amazon, E-Bay, Facebook, Google, Microsoft, and many others, are embracing randomized
experiments to get the best evidence base for their decisions. And while their experiments started
primarily in the digital realm, they are increasingly finding their way into the most important decisions
that these firms make.
X-Lab will demonstrate why experiments – if performed well – can be such an important enhancer of
business decision making. It will also introduce students to the skills they need to run experiments, and
to interpret the results from the experiments of others.
The class will be a hands-on, action-learning experience. Students will work with host firms and non-
profit organizations to design, run, and interpret the results from randomized experiments. They will be
guided in this by mentors assigned to each of the project teams.
Projects that are part of X-Lab continue across semesters, so projects are at different stages. This means
that there will be opportunities, depending on the team they become part of, for students to work on
design issues, implementation, and/or evaluation of the results.
2 of 9
II. COURSE OBJECTIVES
The objectives for the course are as follows:
Understanding how to critically consider different types of evidence for decision making
Understanding the value of running a randomized experiment for decision making in a
business setting
Understanding which types of questions are more or less appropriate for running an
experiment in a business setting. Understanding what the considerations are in designing
such experiments
Learning how to design an experiment and to execute it in a way that takes into account the
practical management and logistical issues facing firms
Learning how to analyze the results of an experiment, and how to be an effective
consumer of other experiments in management research
III. GRADING AND REQUIREMENTS
This class can only be taken for a grade. The grading is divided as follows:
35% Class participation
25% Mid-term presentation
5% Project team 360 evaluation
35% Final class presentation and group project write up
Details:
a. Class Participation (35%)
The character of the course naturally lends itself to active exchange among team members, thus
we encourage, value, and recognize in-class contribution. Effective participation includes
attendance, preparation, and making an active and constructive contribution to discussions
You cannot contribute to the class or your team when you are not present. As such,
both lateness and absences will count against your in-class contribution grade. If you
must miss a class, please let the TA (and instructors) know beforehand.
b.
Mid-term presentation (25%)
Project teams will make presentations in week 6 of their proposed experiment for feedback from
the group. Details provided in the description of Week 6.
3 of 9
c.
Project team 360 evaluation (5%)
Students will also evaluate the contributions of the other team members at the end of the class with
a 360 evaluation.
d.
Final class presentation and group project write up (35%)
All teams will present their projects in the final class of the semester. Not all projects will have final
data to present, but we expect an update of the project as it stands, challenges, etc. Each team member
is expected to contribute to this presentation. In addition, each group is expected to hand in a ~10
page write up of their project.
IV. TEACHING ASSISTANCE AND HELP SESSIONS
The teaching assistants are available to conduct individual or group “help” sessions on an occasional
basis for any students who might find them useful. You should feel free either to approach the teaching
assistants or to make an appointment to see the instructors if you have any questions regarding the
course or the projects. The teaching assistants will also be available during the ‘action learning’
sessions where the teams will be meeting every week to discuss their projects to assist with problems
faced in the projects.
4 of 9
Detail:
Weeks 1: Introduction to experiments
February 7
Students will be introduced to the experiments for the semester.
Homework: Submit order of preference for experiments
Weeks 2-5: Project team meetings
February 14 – March 14
Students will be expected to attend the weekly 3 hour sessions. This should include a project
meeting, but can also include working time. The professors and TA will be on hand to assist
with any problems, and will expect to hear an update from each team on how their project is
progressing.
Week 6: Group presentations
April 4
Groups will present their projects, including:
o
the question they are asking
o
the intervention(s) they are proposing (i.e. for treatment, control, etc.)
o
the sample size,
o
a project management plan and timeline
o
biggest challenges
Opportunity to get feedback from the whole group before launching into the field the
following week
Weeks 7-11: Project team meetings
April 11–May 16
Students will be expected to attend the weekly 3 hour sessions. This should include a project
meeting, but can also include working time. The professors and TA will be on hand to assist
with any problems, and will expect to hear an update from each team on how their project is
progressing.
Week 12: Final project presentations
May 23
Presentations should include:
o
Discussion of why a randomized experiment is the right way to answer this question
o
The question they have asked
o
A theory of change
o
A project design (including a power calculation),
o
Any management challenges
5 of 9
o
Their data collection protocol
If the data from their experiment is in, results should be presented.
If not, the proposed data analysis should be discussed.
6 of 9
X Lab Project Travel Related Policy
X Lab and the project hosts will cover all project related travel costs. Please see ‘Project-
Related Travel Policy – Expenses’ and ‘Project-Related Travel Policy – FAQ’ in the ‘Travel
expenses’ folder in stellar.
X Lab Data Confidentiality Policy
X Lab is sensitive to its hosts’ data confidentiality needs, and students are required to follow
our non disclosure agreements, and other policies. Please see stellar for more information, in
‘Data Management Policy’ folder.
MIT Sloan Policy on Classroom Behavior
In order to create a productive learning environment and to ensure mutual respect it is essential
that the norms and rules of classroom etiquette and behavior reflect the highest standards. It is
also important that these norms be consistently enforced by the faculty across all classes.
Although in the final analysis each faculty
member is responsible for his or her own classroom, there are significant negative consequences
for other faculty and for the School if rules are not consistent and are not enforced. Therefore it is
the policy of the MIT Sloan School that
Students are expected to arrive promptly on time and to stay for the entire class.
Faculty are expected to begin and end class on time.
Laptops and e-readers are not be open in the classroom except with explicit permission of
the faculty (e.g., when used as part of the instructional program or when required by
students because of physical or other challenges)
Cell phones and PDAs are not to be used or permitted to ring in the classroom.
Students are expected to attend all classes.
It is expected that faculty will articulate how these rules apply in their class as well as how
the rules will be enforced.
Dostları ilə paylaş: |