CS442 Course / Workspace / Task Brief
Lab service 00:00
CS442 · Learning Materials

Lecture Slides and ICA4 Guide

Available before sign-in, provided by the course website. Resume your group experiment when the backend reconnects.

Four lecture slide decks

ICA4 workflow · about 70 minutes

  1. 0–5 minutes Read the guide, record your original hypothesis, and identify the access scope and budget.
  2. 5–25 minutes Complete the soft-label loss task and check it with the provided record-alignment helper. Run model extraction at budgets of 50, 200 and 800, then formally evaluate all three budgets.
  3. 25–45 minutes Complete the membership-scores and confusion-count tasks and check them with the provided shadow-calibration helper. Explore thresholds on shadow data, query both target models, and formally evaluate four membership inference combinations.
  4. 45–60 minutes Discuss measured results with the assistant and record an interpretation with a limitation in Lab Notes.
  5. 60–70 minutes Complete the analysis, submit your conclusion and actual filenames, and download evidence.

Each group advances its own stage through formal evaluations and submission.

Preparation and Reflection

  1. Model extraction measures whether a surrogate can copy target behavior. Predict how a larger budget will affect agreement.
  2. Distinguish agreement from task accuracy and explain why they may change differently.
  3. Membership inference asks whether a sample was used for training. Compare four combinations: baseline/regularized models and confidence/negative loss scores.
  4. Calibrate thresholds on public shadow data and observe the TPR/FPR tradeoff.
  5. Use your group metrics, model comparisons and experimental limits in the conclusion. You can organize questions and initial judgments in your own document first.

When the backend returns, save your prediction in the Experiment Workflow, run the experiment, and check your explanation against the results.