Blog Topics

This platform aims to systematically curate and preserve my favored Python techniques for future reference. This digital journal, accessible globally wherever an internet connection is available, is a convenient and effective learning tool. 

Going down a wooded path at night, alone, with your trusted companion by your side, can be scary and daunting.  However, with support from your friends, documenting your path (curating a code base like a blog), and continuously working on the code base can make the process appear as a walk in the park! 




In a world dominated by sophisticated language models like OpenAI's GPT, blogging may seem like a relic of the past. But here's a nuanced perspective: this blog serves as an innovative instrument for both teaching and mastering new programming languages. It's about immersing in syntax, applying it, and then translating the information into layman's terms for broader understanding.

Each topic presented here is elaborated in a separate blog post, supplemented by a Google Colab demo to visually illustrate the subject matter. Topics without links are currently on my exploration radar, awaiting their turn to be dissected and discussed.

The approach I employ for data analysis is essentially a four-step cycle:

1. Import data
2. Clean data
3. Analyze data
4. Export results

Each stage offers a rich canvas for efficiency, clever shortcuts, and a dash of creativity, making data analysis an ever-evolving art form.

Of course, topics arise wherever my muse takes me based on current projects, an interesting article, or based on my current book of interest.  Topics are grouped in a logical order either by area of interest, content, project, or any other perceived natural order.  They don't follow a particular order, but I keep some semblance of reasonableness for the reader.  

Import Data
    pd.read_csv
        
        Blog - Automating Column Name Cleaning with Pandas        

Data Creation
    Workout data by month, day, and hour with randomized reps
        blog      Google Colab
    Course Data - generate course data with prefix, number, level, and location.
        blog      Google Colab
    5k Data
        blog    Google Colab - Generate 5k race data over multiple years, increase age for retained              runners, drop 30 percent of last year's runners, and add 100 new runners a year. 

Exploratory Data Analysis (EDA)
    pd.describe and pandas-profiling
        blog

Cleaning/Preparing Data for Reporting (90% of my time) - np refers to the numpy package while pd refers to the Pandas package. 
    np.select - Create a new variable based on data from another column.
        blog        Google Colab

    pd.cut - Create bins in three different ways.
        blog        Google Colab     
        
    pd.cut and pd.map - I love these functions!
        blog        Google Colab

   pd.dropna
        blog - Google Colab

    pd.fillna
        blog - Google Colab

    pd.drop_duplicates - Unduplicate a list by max value with a prior filter - DataFrame
       blog - Google Colab

    pd.idxmax - find the max value of an index and return the index value
        blog - GoogleColab

    pd.map - recode data with series - map one value to another.
        blog - Google Colab

    Masking - filtering data columns
        blog
        
    pd.split - Create new columns based on a delimiter within another column
        blog - Google Colab (chaining and DataFrames)

    pd.str Access - All things pd.str...
        blog - Google Colab
        
    pd.query - filter DataFrames
        find()
        rfind()

    np.where - create a new column based on data from another column
       blog - Google Colab

    CategoricalDtypes - Explicitly sort a pd.Series based on a defined list
        General - blog - Google Colab
        astype - blog - Google Colab
        Sort Data - blog - Google Colab
        Filter Data - blog - Google Colab

    merge- combine two or more DataFrames row-wise
        blog - Google Colab

    Select Columns in Pandas
        blog - Google Colab

Reporting

    Stylizing a DataFrame with bar - embed a bar chart in a DataFrame
        blog - Google Colab
        blog - More style, more sun!

    pd.rank - Rank top-ten-grossing movies
        blog - Google Colab
          
    pd.group_by and agg
         blog - Google Colab

    crosstab - recreating PROC FREQ in Python
         blog - Google Colab

Reportlab - creating PDFs with a tremendous amount of precision
        blog via Google Blogger - Introduction
        1) pinpointing items on a single page - blog
        2) interacting with tables - blog
        3) handling graphics
        4) managing platypus items that dynamically extend over pages - blog

Report all North Carolina Universities
    blog - Google Colab

My Plotly Adventure 
    My Billboard Example (My learning sandbox, so it is always under construction )
    Introduction - blog
    Callbacks - blog

Matplotlib
    Automation
        Scatter 
        Bar

Environment Management with MiniConda - blog
    Update Pandas to 2.x - blog
    Cloning an Existing Environment with MiniConda - blog
    Importing and Exporting Environments with MiniConda - blog
    Install, update, list, packages, and misc conda environment scripts - blog

    Updating a Module without resetting the kernel - blog!

Fun with IF Statements
    Magic 8 ball and Restaurant Selector - blog - Google Colab

Try and Except
    Exploring Try and Except for a Data Analyst - blog
    An example in context - blog

Python Comprehension - A blog is only complete with Comprehension!
    General Types - blog
        List Comprehension I - blog
        List Comprehension II - blog
        List Comprehension If Else If - blog
        List Comprehension if Else if, Multiple Conditions - blog
        Combine Lists to create a DataFrame - blog

Fun with Lists
    blog - Google Colab - general
    blog - Using lists for rankings in chaining and DataFrams

Python for Loops
    Introduction - for Loops - blog
    Example with a bar chart - blog

Comparing Categorical Data
    Contingency Tables - blog

Series Methods
    Aggregation Methods - blog - Example in Google Colab 
    Filtering np.array - blog
    Filtering np.array II - blog
    Fun with str - Google Colab

Functions
    Introduction - blog - Movies Example in Google Colab
    Introduction II - Arguments - blog
    Introduction III - Type Hints - blog

Pivot_Table
    Intro - blog - Google Colab

np.array play space - blog

loc and iloc (why not) - blog

f'{string}' - way too useful not to cover! -
    blog - General
    blog Google Colab - if/elif examples

Different ways to create a column - Google Colab

Index Manipulation with np.Series - Google Colab
Index Manipulation with np.Series 2 - Google Colab

Data Storage

DuckDB
    blog - introduction - escaping the CSV, basic.
    blog - loop through CSVs and add to DB.

Projects

I have evolved from learning about Python and posting short blogs to project-based applications - I'm applying what I have learned to a complete project.  Below is a list of projects with supporting documentation.  They follow my muse, are analytics-based, and are hobby projects.  They are not linked with my primary role but are designed to continue learning in a more complex construct - a project.  Each project is given a title and then a short description. 

Grants_dot_gov - extract and analyze grant opportunities from Grants.gov. 

5-Year Journal Template - Create a 5-Year Journal PDF with ReportLab.

Comments

Post a Comment

Popular posts from this blog

Drawing Tables with ReportLab: A Comprehensive Example

Image
Tables play an integral part in visualizing and summarizing complex data in a clear, readable format. ReportLab offers a powerful object called `Table` for rendering data in table form within a PDF. By pairing this with `TableStyle`, users can customize tables to fit any styling requirements. Let's break down an example to understand the power and flexibility of ReportLab's table creation capabilities. The first step is to know your table and cast it to a list of list.  I didn't know how to do this off the top of my head, so some quick searching gave me the results! It is a common task for ReportLab users so there are many example out there.  Here is my example:  #Start working with the table budgetinlist = [budget.columns.tolist()] + budget.values.tolist() # Format numbers with commas for i in range(1, len(budgetinlist)):     for j in range(1, len(budgetinlist[i])):         try:             budgetinlist[i][j] = "{:,.0f}".format(float(budgetinlist[i][j]))        
Read more

DataFrame groupby agg style bar

Image
 The goal of the article is to investigate the bar function through the style method of Pandas DataFrame.  So, when we work with DataFrames, we can create a visual within a DataFrame.  What does that mean? We can embed bar charts, sparklines, and mini bar charts in the DataFrame.  This can reduce the amount of cognitive load when reviewing a DataFrame.   Google Colab link with all the code To get started, we are going to import the data: import pandas as pd import numpy as np import pandas_profiling as pp loc = 'https://raw.githubusercontent.com/aew5044/Python---Public/main/movie.csv' m = pd.read_csv(loc) pd.set_option('display.max_columns',None) pd.options.display.min_rows = 10 Next, I want to create a new DataFrame that groups by the rating (e.g., "R" "PG-13"), then calculates the total sum, min, max, and total observations for gross.  content = ( m .groupby('content_rating') .agg({'gross':['sum','min&#
Read more