For example, if we take the following code:

    def test_more_complex(self):
        student = ts.Unit(
            "Student", attributes=[]
        )  # object type, specify data types through object type
        race = student.nominal("Race", cardinality=5, exactly=1)  # proper OOP
        ses = student.numeric("SES")
        test_score = student.nominal("Test score")
        tutoring = student.nominal("treatment")
        race.associates_with(test_score)
        student.associates_with(test_score)
        race.moderate(ses, on=test_score)
        design = ts.Design(dv=test_score, ivs=[race, ses])
        gr = design.graph
        print(gr.get_nodes())
        self.assertTrue(gr.has_variable(test_score))

the print will print an empty list, and the assertion will fail. This seems to be because graph.py requires relationships from tisane.og_variable instead of from tisane.variable, and tisane.design calls tisane.graph.Graph.add_relationship to add edges:

# from tisane/graph.py
    def add_relationship(
        self, relationship: Union[Has, Treatment, Nest, Associate, Cause, Moderate]
    ):
        if isinstance(relationship, Has):
            identifier = relationship.variable
            measure = relationship.measure
            repetitions = relationship.repetitions
            self.has(identifier, measure, relationship, repetitions)
        elif isinstance(relationship, Treatment):
            identifier = relationship.unit
            treatment = relationship.treatment
            repetitions = relationship.num_assignments
            self.treat(unit=identifier, treatment=treatment, treatment_obj=relationship)
        # ...

The types for relationship are imported from tisane.og_variable, which means that none of the relationships are added as edges.

• causes and associates edges should be the same style (solid) and different from has (dotted), nests (dashed)
• dependent variables should be filled in with "light-ish" grey (i.e., grey!30) --> def get_causes_associates_tikz_graph(self) --> def get_causes_associates_tikz_graph(self, dv: AbstractVariable)

12.03: Start trying SAT formulation using Z3

TODO:

• Try ASP -- see which one is easy, etc.

Other:

• [12.03] talked w Alan Borning about possible solvers and logical formulations --> SAT seems to make a lot of sense, finite domain constraints

• Rewrite graph inference rules to not use SMT
• Add tests for using graph to infer main, interaction, and random effects and family/link functions
• Add/debug functionality for constructing graphs from interaction/moderation effects that are specified in the language (see https://github.com/emjun/tisane/commit/a541ba5db952fd4856fc75943e811206e04b5889)

This branch introduces API redesigns/re-implementation, specifically:

• Introduce three data types: Units, Measures, and SetUp variables
• Updated the API to follow more OOP conventions so that measures must be declared through a unit. This enables us to enforce more valid programs/declarations at the API level, effectively enforcing automatically/removing the need to manually check for data measurement relationships between units and measures (was previously another step of compilation)
• Add more tests for the new API
• Start implementing new graph inference rules (this should have been a separate branch. Further development will occur on a separate branch.)
• Update README to reflect the above changes.

KB experimental branch: Tried ASP and SMT implementations in parallel. Decided to stick with SMT implementation. Passes tests. Iterated/revised KB multiple times.

Done experimenting with Knowledge Base implementations for now. Time to experiment with API.

This was a bug.

Seems that issue was that we were deepcopying graphs and nodes when updating the graphs, so the references to objects in the test cases were "older" objects that we updated but were not in the ConceptGraphs. I removed deepcopies in the latest commit.

Goal: Create an R version of Tisane

Considerations:

• Keep user-facing API as R-idiomatic as possible
• Reduce duplicate maintenance efforts. So that changes to Tisane in Python will improve/update Tisane R package.

Pipeline at 10,000 ft

R API (user input code) --> Python script --> JSON --> Tisane GUI --> R Code (output statistical modeling code)

Note: The new part is R API --> Python script. The rest is already how Tisane (Python implementation) works. Put another way, the goal is to "transpile" R into Python.

How to compile/transpile R into Python?

• Strategy 1: Build up internal graph IR in R, traverse graph to produce Python code
• Strategy 2: Parse R script into AST, traverse AST, generate Python code from AST
• Strategy 3: Build up internal graph IR in R, output graph IR in some format (maybe DOT or something like that), read in graph output, write Python code from graph

In all of these: Key thing is to control Python script execution through a bash script, which we can call from R.

Current/next steps

As of January 18, 2022: I opt for Strategy 1 first because (i) I suspect the syntax of Tisane is likely to change more than the graph IR and (ii) outputting the graph to read it back in might not be necessary.

TODOS related to Tisane R:

• @emjun: write R API --> Python script
• @audreyseo: revise code generation (Tisane GUI --> code) (see related issue)

Tisane currently provides two types of conceptual relationships: causes and associates_with. This doc covers when and how to use these verbs.

If a user provides associates_with, we walk them through possible association patterns to identify the underlying causal relationships. In other words, associates_with indicates a need for disambiguation to compile to a series of causes statements.

To do this well, we need to resolve two competing interests: causal accuracy and usability. Prioritizing causal accuracy, the system should help an analyst distinguish and choose among an exhaustive list of possible causal situations. However, doing so may be unusable because the task of differentiating among numerous possible causal situations may be unrealistic for analysts unfamiliar with causality. These concerns do not seem insurmountable.

With an infinite number of hidden variables, there are an infinite number of possible causal relationships. We could restrict the number of hidden variables an analyst considers. This decision compromises causal accuracy for usability. If we had a justifiable cap on hidden variables, it may be worthwhile to take this approach.

Another perspective: If the goal is to translate each associates_with into a set of causes, why provide associates_with at all?

The primary reason I wanted to provide both was because of the following:

• Analysts are sometimes unsure about the causal edges in their conceptual models. This uncertainty can be due to their own lack of knowledge or because the relationships are hypothesized but not known and now the analysts want to see if data supports the hypothesized relationships.
• There may be a lack of definitive evidence in a domain about some causal edges and paths (that may involve multiple variables).

In all these cases, it seems important to acknowledge what is known, what is hypothesized/the focus of inquiry, and what is asserted for the scope of the analysis. (accurate documentation, transparency)

In the current version of Tisane, analysts can express any relationships they might know or are probing into using causes. If analysts do not want to assert any causal relationships due to a perceived lack of evidence in their field, they should use associates_with. Whenever possible, analysts should use causes instead of associates_with.

Tisane's model inference process makes argubaly less useful covariate selection recommendations based on associates_with relationships. Tisane looks for variables that have associates_with relationships with both one of the IVs and the DV. Tisane suggests these variables as covariates with caution, including a warning in the Tisane GUI and a tooltip explaining to analysts that associates_with edges may have additional causal confounders that are not specified or detectable with the current specification.

For the causes relationships, Tisane uses the disjunctive criteria, developed for settings where researchers may be uncertain about their causal models, to recommend possible confounders as covariates.

We assume that the set of IVs an end-user provides in their query are the ones they are most interested in and want to treat as exposures.

What happens if the initial choice of variables could lead to confusion in interpretation of results? We currently treat each IV as a separate exposure and combine all confounders into one model. In some cases, this may lead to interpretation confusion. For example, if the model includes two variables on the same causal path, one of the variables may appear to have no effect on the outcome even if it does (due to d-separation). We currently expect analysts to be aware of and interpret their results accurately in light of their variable selection choices. In their input queries, analysts should include only the variables they absolutely care the most about in their queries.

Moving forward

I would like to see the following (working list, no priority given yet):

• Tisane: Separate out use cases and provide language constructs for each: lack of knowledge vs. hypothesized causal edges vs. lack of definitive evidence in the domain.
  • Language design: Remove associates with, require only causes
  • Provide a "gallery" or "library" of canonical graph shapes/statements they could adapt.
  • Allow for inclusion of hidden variables?
  • Generate multiple linear models to verify the input DAG/mechanism validation.
  • Enforce variable selection that guarantees accurate inference, not just DAG/mechanism validation.
• A question I keep coming back to: How usable is causal modeling to non-experts and how can we make it more usable to them?
  • Study/find out what makes stating "causes" statements difficult for researchers and how to constructively support their skepticism rather than allowing them to avoid formalizing their knowledge.

Implementation changes:

• [BIG] Thomas R. had some hesitation about the theoretical soundness of the disjunctive criteria. He did not expand much, but I hope to meet with him early in the winter quarter to discuss.
• I could re-implement Tisane in R so that it uses Daggity under the hood. Would have to see how to use Daggity under the hood in Python.
• In R, I've never created a widget/plug in, but I can look into how to do that.
• Both R and Python versions could use a code-only interface, not having to rely on the GUI.

Follow-up work/Paper ideas:

• Eval and improve conceptual modeling language
• Eval Tisane vs. R

import tisane as ts
import pandas as pd
import os


FILE_NAME = "schools.csv"

dir = os.path.dirname(__file__)
df = pd.read_csv(os.path.join(dir, FILE_NAME))


# Initialize Units
school = ts.Unit("schid", cardinality=10)
student = ts.Unit("stuid", cardinality=96)


homework = student.ordinal("homework", order=[0, 1, 2, 3, 4, 5, 6, 7])

# school variables
school_size = school.ordinal("scsize", order=[2,3,4,6])
school_region = school.nominal("region")
school_type = school.ordinal("sctype", order=[1, 4])
public = school.nominal("public")

# Define relationships
public.causes(homework)
public.moderates(school_region, on=homework)
school_size.associates_with(homework)
school_type.associates_with(homework)

design = ts.Design(dv=school_size, ivs=[school_region]).assign_data(df)

ts.infer_statistical_model_from_design(design=design)

In API_OVERVIEW.md, the description of the effect parameter to causes is effect: tisane.variable.AbstractVariable -- the cause data variable. Is effect not supposed to be the the result of the cause-ing variable?