Causal Analytics for Applied Risk Analysis Louis Anthony Cox, Jr
Yüklə 12,64 Mb.
|
- Bu səhifədəki naviqasiya:
- Using Causal Models to Evaluate Total Effects vs. Direct Effects
- Using MDPs and DES Causal Models to Evaluate Policies and Interventions
- Causality in Learning Analytics
- Analytics Task Models for delivering results Principles, methods, algorithms
- Causality in Collaborative Analytics
- Causal Models in Game Theory: Normal and Extensive Forms, Characteristic Functions
- Causal Models for Multi-Agent Systems
- Conclusions: Causal Modeling in Analytics
- Causal Model Controlled inputs (decisions) Random outcomes
- REFERENCES FOR CHAPTER 1
|
The bottom half of this table is redundant since the conditional probabilities for the outcomescure and no cure must sum to 1 for each set of input conditions, but the values for no cure are included in the CPT for completeness. Problem: Calculate the population-level effect of an intervention that changes from treating all patients with the old treatment to treating all patients with the new treatment at a hospital in which half the patients are men and half our women. Also calculate the individual-level effects for men and for women. Solution: The population-level effect can be calculated as follows. Define the outcome variable to have value o = 1 for a patient who is cured and o = 0 otherwise. Then the change in the fraction of patients cured caused by the intervention is: E(o | a  ) - E(o | a) = E(o | new treatment) - E(o | old treatment)
= = = = = (1 - 0.5)*0.5 + (0.1 - 0.5)*0.5 = 0.25 - 0.20 = 0.05, where in the last line, appropriate conditional probability values from the CPT and the assumed marginal probabilities of 50% women and 50% men are substituted for the symbolic expressions. The net result is that changing from the old to the new treatment increases the average cure probability by 5% if the composition of the treated population remains half men and half women. Rational and informed individuals will care more about individual-level outcome probabilities than about population averages. In this case, the individual-level effects for men and women were averaged in the last line of the above calculation using the fractions of the patient population that were men and women, respectively. The individual-level effects were that the intervention improved the cure rate for men by 0.50, from 0.50 with the old treatment to 1.0 with the new; but it reduced the cure rate for women by -0.40, from 0.50 with the old treatment to 0.10 with the new treatment. Presumably, such an intervention would only be made for the whole population of patients if it were not yet realized that it harmed women. A better individual-level intervention would be to apply the new treatment only to men. Using Causal Models to Evaluate Total Effects vs. Direct Effects From the perspective of causal modeling, the best way to use data to estimate the causal impact of an intervention on an outcome is often to estimate and validate a causal modelfrom the data and then use it to quantify how much difference an intervention makes for affected populations and individuals. In general, the effects of policy changes on outcomes of interest can be predicted and evaluated correctly only by modeling the network of causal relationships by which effects exogenous changes propagate among variables. Figure 1.11 illustrates this point via a directed acylic graph (DAG) model in which income has not only a direct causal effect on health_outcome, signified by the arrow between them, but also indirect effects transmitted via pathways that include residential location and exposure. Fig. 1.11 Direct and indirect paths in a causal DAG model network income residential_location health_outcome exposure Predicting how health_outcome probabilities would change if income were exogenously changed requires quantifying the effects transmitted via these multiple pathways. This cannot be accomplished by statistical methods that ignore the network of relationships among variables, e.g., by regressing health_outcome against income and other variables. Such a regression model does not predict how an exogenous change in income would change the joint frequency distribution of values for residential_location and exposure in the affected population. But these changes must be known to predict how the distribution of health_outcomein the population will change, since it depends on them. Therefore, methods that go beyond regression modeling are needed to quantify how exogenous changes in one or more variables will affect the joint probability distribution of the remaining variables and, specifically, the distribution of outcomes of interest, such as health_outcome. Causal network analysis methods address this challenge. Moreover, they provide useful distinctions among different kinds of causal effects. For example, the total causal effect of a change in income on health_outcome is the change in the probability distribution of health_outcomethat occurs when changes in income propagate via all pathways, causing changes in residential_location and exposure that, in turn, contribute to changes in the conditional probability distribution of health_outcome. By contrast, the direct causal effect of a change in income on health_outcome is the change in the probability distribution of health_outcome that occurs when changes in income propagate only via the direct link from income to health_outcome. Thus, to compute the direct causal effecton health of an intervention that exogenously changes income, the CPT for health_outcomewould be used to calculate the difference in the conditional probability distribution for health_outcomemade when only the value of income changes from its old to its new value. By contrast, the total causal effect of the intervention is found by using the CPT for health_outcome to compute how its conditional probability distribution changes when the distributions of all variables affected by the change in income also change as it changes, in accord with their own CPTs. Chapter 2 introduces algorithms for evaluating total and direct causal impacts of interventions on outcomes in DAG models using software to facilitate the calculations. In general, no single number, such as a regression coefficient, can represent both the direct and the total effect of a change in an explanatory variableon a dependent variable. Causal network analysis allows such distinctions to be drawn with clarity and provides straightforward ways to define and evaluate both kinds of effects, as well as related concepts such as indirect effects and effects mediated by a specific variable or pathway. Chapter 2 explains these additional concepts and shows how to evaluate quantitatively each type of effect in DAG models. The same ideas and principles carry over to more sophisticated causal models, such as dynamic simulation models. Using MDPs and DES Causal Models to Evaluate Policies and Interventions Evaluating the effect of a past action, policy, or intervention on observed outcomes is in general no easier than developing a full causal model for how actions affect outcome probabilities: they are essentially the same problem. Understanding how past actions have affected current outcomes (or their probabilities, if the causal relation is not deterministic) requires essentially the same causal analysis techniques as predicting how current actions will affect future outcomes or their probabilities. When enough knowledge and data are available to develop and validate them, discrete-event simulation (DES) models and Markov decision process (MDP) models can be very useful for evaluating effects of past programs and comparing the effects and the cost-effectiveness of alternative polices, treatments, and interventions. They have been widely used for this purpose in healthcare program evaluations. For example Goehler et al. (2011) survey the results of 27 Markov models, 3 discrete-event simulation models, and 4 mathematical models taken from the literature on evaluation of different interventions to reduce risks of chronic heart failure (CHF). Interventions evaluated by these models included efforts to improve screening, diagnosis, treatment with drugs or devices, disease management programs, and a study of heart transplants. The dynamic simulation models are able to simulate how probably distributions of outcomes over time in a treated population shift in response to different interventions. Together with implementation cost and timing information, this provides the crucial information about the probable consequences caused by different interventions needed for well-informed decision-making based on methods such as cost-effectiveness and risk-cost-benefit analysis. Simulation-based evaluations of effects of interventions are vulnerable to misuse if the simulations are not based on valid causal models. Comparing probability distributions for outcomes under real and hypothetical conditions can lead to quantified “effects” estimates that cannot be verified and that rest on the validity of the hypothesized conditions, which may be unknown. Different hypothetical modeling assumptions could produce different answers. Chapter 2 discusses further counterfactual and potential-outcomes causal modeling. Causality in Learning Analytics The preceding sections describe how causal analytics and causal models can be used to describe and interpret patterns of associations among observed variables, predict outcome probabilities for different decisions or for the status quo, optimize decisions, and evaluate how interventions have changed or will change outcome probabilities. Table 1.6 summarizes models and methods for carrying out these tasks, as discussed in this chapter and Chapter 2. Table 1.6 Models and methods for analytics tasks
The causal DAG models (including influence diagrams) and simulation models shown in the middle column can support the analytics tasks shown in the left-most column if the required models are known. But suppose that no trustworthy causal model of how outcome probabilities depend on decisions is available when decisions must be made. Then, how should a decision maker proceed? This commonly happens when the system or situation of interest is not yet understood well enough to credibly simulate or model how it will (probably) respond to changes in inputs. Learning from experience how to act effectively in such an uncertain world is the central challenge for learning analytics. The usual context for machine learning algorithms that address this challenge is a sequence of repeated opportunities to choose an act, perhaps informed by accompanying observations, resulting in outcomes to which rewards are assigned (Sutton et al., 1992). The decision maker (d.m.) makes a sequence of decisions, such as which treatments to prescribe to patients with certain medical histories; which loan applications to approve; how much inventory to order each day; or how to adjust the position of an autonomous vehicle on the road. Outcomes are gradually revealed: patients improve or worsen or die, borrowers repay their loans or don’t, vehicles stay on the road or stray off it, and so forth. Some of these outcomes earn rewards or penalties; generically, we use “rewards” to refer to the values assigned to the outcomes, with larger rewards being preferred. The d.m. seeks to make decisions to maximize an objective function such as average reward per unit time, total reward by a termination time, or discounted reward. This set-up is, of course, extremely reminiscent of those for statistical decision theory, MDPs, POMDPs, and simulation-optimization. The big difference is that now, since no causal model is initially known, the d.m. must act on, and learn from, the real-world environment rather than from a model of it. This implies that certain outcomes, such as patient deaths or autonomous vehicle crashes, which might be informative in a simulation should be avoided as well as possible while learning. This is an aspect of the famous exploration-exploitation tradeoff between choosing to reap the rewards from what are currently expected to be the best or safest decisions based on experience so far; or instead to take some risks to discover whether other decisions might yield even better rewards. The following approaches can help to decide what to do next in such settings, even in the absence of a causal model.
Q(s, a) Q(s, a) + (R + Q(s’, a’)-Q(s, a)) (1.12) In words, the estimated value of taking act a in state s, denoted by Q(s, a), is updated by taking its current value on the right side of the update arrow and adding an increment proportional, with proportionality constant , to the difference between the estimated experienced reward, R + Q(s’, a’), and the reward actually received. Under certain comditions, such as that each state of the MDP is visited infinitely often as the time horizon for the MDP becomes infinitely long (no absorbing states), the SARSA updating rule leads to the same optimal policy as the Bellman equation for stochastic dynamic programming, but with the considerable practical advantage that the rewards and transition probabilities need not be known initially: SARSA learns the optimal policy via well-chosen trial-and-error learning. Reinforcement learning software is available at these sites:
MAB software for computing the probability of each action being best after any history is available at the following link:
These methods for deciding what to do when no causal model is available can be combined and extended in various ways. Doing so has led to a recent burgeoning of productive new methods for adaptive decision-making and learning in artificial intelligence and machine learning. For example, the UCB1 algorithm has been combined with Monte Carlo tree search (MCTS), yielding the “UCT” (UCB1 for trees) algorithm for learning to play games. UCT has been streamlined and further improved by incorporating value-of-information (VOI) heuristics, and has been applied successfully to multi-armed bandits, Markov decision processes, and adversarial games (e.g., Tolpin and Shimony, 2012; Pepels et al., 2014). The original multi-arm bandit problem has been generalized to settings where the reward distributions from different actions are not independent, are correlated with observable information (“contextual bandits”), change over time (“restless bandits”), or are manipulated by an intelligent opponent or attacker (“adversarial bandits”). Deep learning methods for succinctly describing and estimating state-action values have been combined with variations on SARSA by several investigators (e.g., Ganger et al., 2016; Van Seijen et al., 2009). There are many other examples of the confluence of technical ideas leading to ever-more powerful and robust optimization of decisions without models. These procedures are so effective that they are often used for simulation-optimization when models are known\. Both adaptive optimization of action-selection probabilities and EVOP-like design-of-experiments and iterative improvement of estimated statistical models and optimization of decisions implied by the estimated models are widely used in current simulation-optimization algorithms and software (Amaran et al., 2016). The success of these model-free algorithms for adaptively learning to optimize decisions raises the questions of why and whether causal models are really needed in analytics. After all, if a learning algorithm can learn fairly quickly to make effective decisions without them, then why use them? Answers include the following.
https://cran.r-project.org/web/packages/causaleffect/causaleffect.pdf.
Thus, despite the exciting advances in machine learning applied to efficient discovery of low-regret decision rules, causal modeling and inference methods have valuable roles to play throughout these other parts of the risk analytics sequence. Causality in Collaborative Analytics Commercially available collaborative analytics information technologies (IT) platforms enable multiple data collectors, analysts, and decision-makers to share data and insights, both throughout an organization and in physical or virtual “war rooms” and operations centers where information is displayed, analyzed, and acted upon by teams collaborating in real time. Causal analytics models can provide useful content and organizing structures to coordinate the collection, analysis, and display of information on such plaforms. Causal graph models such as causal Bayesian networks and influence diagrams provide useful conceptual and visual aids to help teams of collaborating analysts combine, coordinate, and synthesize the research and expertise of different subject matter experts (SMEs). They allow SMEs with complementary expertise about different parts of a problem – such as emission reductions technologies, costs, and health effects of pollution in Figure 1.5 – to develop the parts of the diagram and the corresponding conditional probability tables or models that each understands best, while understanding how the pieces fit together. Causal Models in Game Theory: Normal and Extensive Forms, Characteristic Functions More generally, effective cooperation across and collaboration within analytics centers requites principles and insights into how multiple agents can best work together to accomplish shared goals such as managing a complex system – whether a logistics network, a nuclear power plant, a commercial forest or fishery, or a multi-divisional firm – to reduce risks and increase the production of desired outcomes. Many socioeconomic, technological, and business systems are governed by the interacting choices of multiple decision-makers. The decision-makers may having different information, different opportunities to take actions, and different individual preferences and values, beliefs, and incentives. For example, a business’s pollutant emissions may be regulated by a combination of Federal, state, county, and city authorities. It may also reflect past capital budgeting and investment decisions about which production and pollution-abatement technologies to purchase; ongoing operating and production decisions from multiple divisions; and remediation and clean-up decisions by various vendors and subcontractors hired by the business. The confluence of these many decision threads over time affects how much pollution is actually produced. To understand how to improve decision-making by multiple decision-makers in groups, teams, or organizations, it is necessary to extend normative principles for prescriptive analytics beyond the expected utility formalism (Equation 1.1) to allow for multiple agents. Normative principles that seem unexceptionable for individual decision makers need to be rethought for groups of decision makers. For example, an individual decision maker should never choose a strictly dominated strategy, meaning one that always yields a lower-utility outcome than some other feasible choice no matter how uncertainties are resolved. But the notorious Prisoner’s Dilemma shows that in situations with multiple decision-makers and negative externalities, if each decision-maker chooses an undominated strategy, then these choices can cause each to receive lower-utility outcomes than if each had instead chosen a dominated strategy (Thomas, 2003). Cooperation and coordination of individual choices are required to avoid such Pareto-inferior outcomes. With the need for coordination and cooperation come a host of new technical, practical, and conceptual issues, including how to model information and incentives; trust and reputation; credible commitments and signals of intent; asymmetric and private information; altruism and free riding; costly private or public monitoring and enforcement of agreements; and sharing of information, control, and rewards or losses. Game theory addresses these and related topics in detail (Thomas, 2003; Myerson, 1991; Osborne, 2004; Shoham and Leyton-Brown, 2009). It provides both theory and algorithms for fair division of jointly produced gains; clarifies the existence, uniqueness, computational complexity, and computability of various proposed solutions to the joint decision problem for multiple agents (e.g., finding sets of choices such that no agent or coalition can increase its own expected reward by unilaterally changing its own decisions); and establishes impossibility results for design of collective choice mechanisms that simultaneously satisfy several desired properties, such as fairness (e.g., symmetric treatment of participants), efficiency, budget balance, and incentive-compatibility (e.g., voluntary participation and cooperation). In each of these areas, causal models relate the joint decisions of multiple agents to the joint probability distributions of their rewards. These models are essential for analysis and applications. Important special cases that focus on multiple agents cooperating to accomplish shared goals include the mathematical theories of teams and organizations (Marschak and Radner, 1972; Yüksel and Saldi, 2017) and of risk and reward sharing in investment syndicates (Wilson, 1968). Game theory uses causal models that generalize normal form (decision table) and extensive form (decision tree) models from decision analysis to allow for multiple decision makers. These extensions yield payoff matrix models mapping n-tuples of player choices to n-tuples of resulting rewards or utilities,where n is the number of players; and game tree models in which different players are able to choose moves (branches) at different nodes, based on the information available to them at the time. The decision makers are usually called “agents” in the artificial intelligence and machine learning literature, or “players” in classical game theory terminology. They may have different initial information, including information about each others’s utilities, beliefs, intentions, capabilities, constraints, and resources (or, more generally, each other’s “types”), which they update over time by conditioning on their own observations. Private information arises when observations differ for different agents. Their actions jointly determine the probabilities of their individual rewards (often called “payoffs”in the older game theory literature) via the causal model. In addition to normal form payoff matrices and extensive form game trees used in analysis of non-cooperative games, cooperative game theory uses causal models called “characteristic functions” that specify how much reward each subset or coalition of agents can obtain if it acts by itself (Myerson, 1991; Shoham and Leyton-Brown, 2009). In most game-theoretic modeling, the relevant causal models (or “mechanisms”) mapping player choices to their outcome or reward probabilities are exogenously specified, sometimes by a social or institutional mechanism designer attempting to elicit deired behaviors from agents. In evolutionary game theory, there is no centralized mechanism designer, but mechanisms emerge ane evolve from the interactions of agents game theory. Most of the game-theoretic analysis then focuses on what the agents should choose to do, or on what behaviors and decision rules governing agent interactions (including shared norms and conventions) will emerge and persist, given the specified causal relations between choices (or behaviors) and their probable consequences. In practice, causal analytics methods and models applied to available data and knowledge may be needed to create usefully realistic game-theoretic models for supporting predictions and decisions. Causal Models for Multi-Agent Systems The causal models discussed earlier in this chapter for prescriptive analytics also have generalizations for multiple cooperating decision-making agents. For example,
The artificial intelligence, machine learning, and control engineering literatures on cooperative multi-agent control of uncertain systems and on coordination of team actions and communications in uncertain environments are vast. They include many elegant and profound results and frameworks, including exact forms of optimal control laws for each agent in some models and proposed formulations that apply to human organizations (Marschak and Radner, 1972) as well as to engineered systems (Ho and Chu, 1972). Causal modeling contributes several important components to this rich body of knowledge. First, causal models, when they are available, permit simulation (or, in special cases, analytic solutions) for predicting the outputs or output probabilities of a system for different inputs. Simulation, in turn, permits simulation-optimization of system designs, decision rules, and operating decisions to improve performance metrics and make preferred outcomes more likely, even in uncertain environments and even when multiple intelligent agents are involved in the operation or use of the system. Thus, causal models provide a foundation for improving the design and operation of many real-world systems. Second, the study of collaboration in multi-agent systems (MAS) suggests new ways to manage a variey of risks using distributed sensors and controllers. The opportunities to use distributed communication and control systems to monitor and reduce risks and to improve performance in areas from production and logistics to transportation and energy infrastructures to improved healthcare and patient and customer experiences are already enormous. They are increasing as sensor networks, the internet of things, and ubiquitous access to data become increasingly widespread realities. Finally, the study of collaboration among software agents, each collecting and processing local data, updating its own beliefs about the causal relation between its actions (and perhaps also the actions of others) and resulting probabilistic changes in the world it acts on, taking actions based on its own information and beliefs, and communicating results within a team, may yield insights that help to improve distributed analytics and cooperative goal-seeking, control of enterprises and systems, and risk management in human organizations. This was a goal of mathematical team theory more than half a century ago (e.g., Marschak and Radner,1972; Ho and Chu, 1972). Its fruition is starting to be seen today in applications Conclusions: Causal Modeling in Analytics This chapter has surveyed numerous causal models and methods for representing and calculating conditional probabilities of outcomes in response to changes in the inputs to a system or situation. They are summarized in Table 1.7. Their inputs can usefully be partitioned into those controlled by a decision-maker – the decision variables or control variables, usually with values to be set or chosen by the decision-maker to optimize expected utility or some other criterion, subject to feasibility constraints – and those not controlled by the decision-maker. The uncontrolled inputs represent random variables, such as exogenous noise, shocks, or uncertain quantities, that affect the outcomes but that are not selected or set by the decision-maker. Each causal model can be thought if as mapping its inputs into conditional probabilities for outputs, which include outcomes and rewards in all models, and also observations and changes of state in dynamic models. A causal model links actions to their probable consequences and shows how other variables affect each other and the outcomes. Probabilistic dependencies among variables are expressed through conditional probability table (CPTs) or other conditional probability models. Causal models differ from statistical models by clearly distinguishes between effects of doing something (setting a controlled variable to a certain value) and of seeing something (observing that a variable has a certain value): acting on the world can cause the probability distributions of vatiables to change, but observing the world only allows us to update our beliefs about it (Pearl 2009). This fundamental distinction is discussed further in Chapter 2. Table 1.7 Causal models relating decisions to probabilities of consequences
Once a causal model for how actions affect outcome probabilities has been formulated, it can be used for a variety of purposes, including describing how variables depend on each other; identifying the most probable explanations for observations; comparing outcome probabilities under different counterfactual scenarios; predicting probabilities of outcomes for current or assumed initial conditions and decision rules; and recommending courses of action that make preferred outcomes more likely to occur. To extract useful advice from causal models on what to do next, it is necessary to have solvers that use them to determine the feasible settings of decision variables that cause the most desirable probability distribution of outcomes. A variety of algorithms are available for using the causal models in Table 1.7 to identify acts or decision rules that optimize various criteria, such as maximizing expected utility or average reward per period or expected net present value of future rewards; or minimizing expected loss or cumulative regret. Approaches introduced in this chapter include the following:
Adaptive optimization principles such as optimism under uncertainty (e.g., the UCB1 algorithm), probability matching (via Thompson sampling), and “follow-the-perturbed-leader” (FPL) algorithms for on-line prediction and decision problems have proved surprisingly effective in many settings for yielding high-quality, low-regret approximations to the optimal control laws that would be used if valid causal models were known. These adaptive optimization techniques can be viewed as learning causal models on the fly for the effects of acts on outcome probabilities in different situations.
Mastering any of these techniques provides valuable skills for formulating and solving certain types of analytics problems using appropriate causal models. Their learning curves are substantial, however, and thorough understanding usually requires experience applying them. The references at the end of this chapter provide useful points of entry to the relevant technical literatures. Fortunately, modern object-oriented software makes it possible to encapsule much of the detailed knowledge and expertise needed to apply these algorithms correctly to appropriate causal models or data. Causal modeling is used throughout the rest of risk analytics to help describe and interpret data and to predict, optimize, and evaluate the impacts of risk management decisions (Table 1.6). Chapter 2 examines more closely the multiple meanings of “cause” and “causality” and discusses the senses that are most useful for analytics tasks such as prescription and evaluation. It also introduces techniques and algorithms for learning causal models from data and for using them to carry out other analytics tasks. The emphasis is largely, but not exclusively, on causal Bayesian networks (BNs) and other models that can be represented as BNs. Such models turn out to be both relatively tractable to learn from data in many practical applications, and also highly useful for insightful description, prediction, prescription, and evaluation of the effects of decisions and interventions. Chapter 2 introduces the main ideas of causal BNs and related causal modeling methods and describes free software implementations to enable practitioners to apply them.
Yüklə 12,64 Mb. Dostları ilə paylaş:
|
- 
+
