prof: Adam Okulicz-Kozaryn adam.okulicz.kozaryn@gmail.com

office: 321 Cooper St, room 302

office hours: Thu 1-2, and by appointment, or just stop by: this semester I am in most of Wed and Thu

prerequisites

course description and learning objectives/outcomes

data: sources, best practices, tips and tricks; this class is as much about Python as about data: you'll use the data you chose that will serve you well beyond this class!

hands-on/applied Python/Pandas (and some theory)

you will learn how to manage your data: clean, organize and manipulate, eg: data types, text/math functions, recoding, merging, and reshaping

use AI to write code, but edit it and reduce bloat; improve it, play with it, and make sure you understand it

required textbooks and materials

sign up or login at github.com

(depending on os/browser) on top left hit "New" or "Create Repository" or top right under plus "+" select "New repository"

pick some repo name, say "datMan" ; keep selected 'Public'; important!: turn on "Add README"; and hit at the bottom "Create repository"

then hit "Settings" around middle-top right; on the left select "Collaborators" tab and hit "Add people" : "theaok", and hit "Add theaok to this repository"

go to https://github.com/theaok/datManPy/blob/main/pandas.ipynb and hit 'open in colab' OR go to colab.research.google.com and on popup pick GitHub, search for:
https://github.com/theaok/datManPy/blob/main/pandas.ipynb (it should find it and load it into colab)

best project examples

grading

5ps x 20% problem sets [Py notebook in GitHub]; may cowrite code upto 2 people but then the project must be 2x better than a single-authored one

bonus/extra upto 5% class engagement, class participation eg answering/asking questions, helping others, listserv discussions

bonus/extra upto 5% civic engagement (see bottom of the syllabus)

calendar

sep2 intro f26 [old vOld]

ps0.pdf (i open pdf in zoom-able ilovenote.net)

pandas.ipynb

intro.pdf

replication.pdf

sep9 I/O (Input/Output) and basic descriptive statistics f26[ old vid v old vid]

ps1.pdf

again, are you getting listserv emails? no, or not sure? email me! and maybe revisit mechanics from last week: github, colab, etc

ps0 super quick 3min sharp presentations

data.pdf

find_data.pdf

pandas.ipynb

sep16 manipulate data f26 [old vid v old vid]

ps2.pdf

ps0/ps1 easy, but if needed: skim thru ps0 comments on the listserv, how is ps1 going? if time flip the class

manipulate.pdf

pandas.ipynb manipulate; and dive into merge (1st basic example)

sep23 merge f26 [ old vid vold vid]

pandas.ipynb: merge

may do quick presentations of ps1

flip a class work on ps2: (I walk around and sit with each of you; Q and A; otherwise I look at your colabs, and then approach you with ideas)

if time do real world examples from next class

sep30 real world examples and plotly f26 [old vid v old vid v old vid]

ps3.pdf

QaA and go over your ps2, maybe some presentations towards the end

datasets of the week: https://www.ers.usda.gov/data-products; https://www.irs.gov/statistics/soi-tax-stats-migration-data; https://www.njoag.gov/force/

real world data management (eg mapping/recoding urbanicity) example (covid city paper)

real world merge example: Eric; and another example, Xiao

https://colab.research.google.com/github/theaok/vis/blob/main/plotly.ipynb merge is typically a necessary initial step, but usually the final step is to explore the new relationships

bonus/extra: pandas fancy stuff and other fancy stuff (also focus on your projects, discuss, brainstorm, flip the class)

oct7 wrap up pandas and pandas extra topics (slow down and flexible: you chose what to learn) f26 [old vid v old vid v old vid v old vid]

ps4.pdf

go over ps2, go over merge investigation/interpretation again!, QandA on merge, flip the class work on ps3

wrap up all of pandas

extra topics

oct14 ps3 presentations; and profiler, imputations, fuzzywuzzy f26 [old vid v old vid]

present ps3: 10min sharp + 10min discussion; focus on interesting stuff like research question, data, variables, relationships: descriptive stats and visualizations; skip boring stuff like subseting and renaming

profiler, imputations, fuzzywuzzy

>>>note: the following will be updated>>>

oct21 theory and flip the class work on ps4 vid old vid

theory.pdf

continue with last week presentations; and go over listserv ps3 comments

flip the class work on ps4: slow down, focus on your projects, redo/improve/polish

oct28 theory vid old vid

ps5.pdf

factor analysis

theory.pdf: 2nd sec: CS stuff

chetan presentation

nov4 ps4 presentations vid old vid

revisit theory from last class

presentation: no need for slides, just the notebook: 15min sharp + 15min discussion/q and a

nov?? review, q and a vid

Srija presentation

go through code from earlier, focus on whats underused, need more elaboration etc like: missing obs/duplicates (eg profiler), groupy agg, recode/map, merge, imputations, apis/fred

check out my python notebooks for research; you should use the data you produced in this class to write a paper: publish or perish:

pop growth and happiness pdf and colab

covid and happiness pdf and colab

flip much of the class work on ps5/final project

nov?? vid

shell and AI ideas

Eric joins via livestream https://rcit.rutgers.edu/av-request/live/08699-1-2025 and via zoom https://pwa.zoom.us/wc?mn=8892839953&pwd=dFhiTE1BZVlnMXdWSWN6d3N3MXI0QT09

if time: i will pull up your latest and go over it

dec?? wrap up, summarize

vid

wrap.pdf

ad http:theaok.github.io/swb

revisit theory

Srija present sentiment analysis

pull up instructive chunks of code from: ???

i fork couple best repos as example for future classes

dec?? last class ps5/final presentations !!zoom only!! https://rutgers.zoom.us/j/8892839953?pwd=dFhiTE1BZVlnMXdWSWN6d3N3MXI0QT09

15min sharp + 15min discussion/q and a

just to be safe, delete the data you have posted online, you never know: someone may be picky about it

rules

do not share or link to class videos! These videocasts and podcasts are the exclusive copyrighted property of Rutgers University and the Professor teaching the course. Rutgers University and the Professor grant you a license only to replay them for your own personal use during the course. Sharing them with others (including other students), reproducing, distributing, or posting any part of them elsewhere -- including but not limited to any internet site -- will be treated as a copyright violation and an offense against the honesty provisions of the Code of Student Conduct. Furthermore, for Law Students, this will be reported by the Law School to the licensing authorities in any jurisdiction in which you may apply to the bar.

attendance Attendance is required: if you cannot attend without documented emergency you will lose participation credit; either way do let me know ahead of time; we can put you on zoom so you can participate Be advised that you are responsible for any material covered in the class, whether or not it was in the readings or lecture notes. You are also responsible for any announcements made in class. For most students, attendance is simply essential to learning the material. If you do need to miss a class, be sure to consult with a fellow student to learn what transpired.

zoom policy Do not have to have camera on at all times, BUT do turn it on at times, eg when you talk, or at very least be active on chat. Lets plan to discuss and engage every 15min or at least 30min or so: eg I will show an interesting map and lets talk about it, and lets screen-share your stuff, too; and we will have breaks at least 5min every 1hr

no student left behind policy I am emphasizing asking questions, but especially if you struggle, then definitely do ask questions. Lets work together outside of the classroom to bring you up to speed, and do note that you actually get extra credit points for asking questions and working with me outside of the classroom.

AI May actually start with AI: in assignment submit in appendix both prompt and output (and AI name: chatGPT, Gemini, etc), then refine that and submit as assignment. Always be explicit about AI and cite, either whether fully AI writtten or only initial draft. Cannot have more than 90\% of ps done by AI! And ideally should be much less than that. Because everyone-is-using-ai-all-the-time-for-everything we will focus more on oral presentations so that I am satisfied that the student understands thoroughly what is in the ps.

incompletes: Generally speaking, the material in this course is best learned as a single unit. I will grant incompletes only in cases where a substantial change in life circumstances occurs that is beyond the control of the student, and only with appropriate documentation.

study groups. You are encouraged to form a regular study group. Many students over the years have found the study groups to be very helpful. Study groups are permitted and encouraged to work on the problem sets together. However, each individual student should write up his or her own answer to hand in, based on his or her own understanding of the material. Do not hand in a copy of another person’s problem set, even a member of your own group. Writing up your own answer helps you to internalize the group discussions and is a crucial step in the learning process.

academic integrity. I am very serious about this. Make no mistake--I may appear accommodating and informal--but I am extremely strict about academic integrity. Violations of academic integrity include cheating on tests or handing in assignments that do not reflect your own work and/or the work of a study group in which you actively participated. Handing in your own work that was performed not for this class (e.g. other class, any other project) is cheating, too. I have a policy of zero tolerance for cheating. Violations will be referred to the appropriate university authorities.

For more information see http://fas.camden.rutgers.edu/student-experience/academic-integrity-policy

accommodating students with disabilities. Any student with a disability affecting performance in the class should contact the disability office ASAP: https://success.camden.rutgers.edu/success-services/disability-services/

ru diversity statement https://diversity.rutgers.edu/

---

civic engagement component (opportunity for extra credit!)

Start early. Start thinking about how you want to engage civically today. Universities and social science should serve society. You are encouraged have to engage with local community.

The idea is that you engage civically using research methods. There are several ways to do it. Ideally, you will partner with a local organization, obtain data from them, do some analysis, and present results to them. You may also use government data, say from census bureau, and present relevant information to locals. A local organization can be Rutgers research institute such as WRI, CURE, LEAP or any other organization such as school or soup kitchen or CamConnect. Rutgers Office of civic engagement may be able to help you contact them. The key idea is partnership: you will use tools from this class to produce output useful to local community. This is similar to taking a role of an apprentice at a local organization or serving as a consultant.

Using real world data poses challenges, which is a part of exercise. Presenting your findings to stakeholders outside of a class is also challenging. At the same time, it is fairly easy to contribute locally by using simple tools learned in this class. For instance, simple comparison of means between two schools in Camden can be revealing and helpful locally.

An obvious way would be to use data at your workplace or at a workplace of someone you know. However, you need to make sure that it serves society in some way. For instance, it would be straightforward if you work at a hospital or school or fire department; but it would be difficult if you work at Starbucks.