Clean and Modular Code

  • PRODUCTION CODE: software running on production servers to handle live users and data of the intended audience. Note,this is different from production quality code, which describes code that meets expectations in reliability, efficiency, etc., for production. Ideally, all code in production meets these expectations, but this is not always the case.
  • CLEAN: readable, simple, and concise. A characteristic of production quality code that is crucial for collaboration and maintainability in software development.
  • MODULAR:logically broken up into functions and modules. Also an important characteristics of production quality code that makes your code more organized, efficient, and reusable.
  • MODULE: a file. Modules allow code to be resued by encapsulating them into files that can be into other files

Refactoring Code

  • REFACTORING: restructuring your code to improve its internal structure, without changing its external functionality. This gives you a chance to clean and modularize your program after you've got it working.
  • Since it isn't easy to write your best code while you're still trying to just get it working, allocating time to do this is essential to producing high quality code. Despite the initial time and effort required, this really pays off by speeding up your development time in the long run.
  • You become a much stronger programmer when you're constantly looking to improve your code. The more you refactor, the easier it will be to structure and write good code the first time.

Writing Clean Code

Meaningful Names

Tip:Use meaningful names

  • Be descriptive and imply type : E.g. for booleans, you can prefix with is_ or has_ to make it clear it is a condition. You can also use part of speech to imply types, like verbs for functions and nouns for variables.
  • Be consistent but clearly differentiate : E.g. age_list and age is easier to differentiate than ages and age.

  • Avoid abbreviations and especially single letters : (Exception: counters and common math variables) Choosing when these exceptions can be made can be determined based on the audience for your code. If you work with other data scientists, certain variables may be common knowledge. While if you work with full stack engineers, it might be necessary to provide more descriptive names in these cases as well.

  • Long names != descriptive names : You should be descriptive, but only with relevant information. E.g. good functions names describe what they do well without including details about implementation or highly specific uses.

Try testing how effective your names are by asking a fellow programmer to guess the purpose of a functino or variable based on its name, without looking at your code. Coming up with meaningful names often requires effort to get right.

Nice Whitespace

Tip:Use whitespace properly

  • Organize your code with consistent indentation - the standard is to use 4 spaces for each indent. You can make this a default in your text editor.
  • Separate sections with blank lines to keep your code well organized and readable.
  • Try to limit your lines to around 79 characters, which is the guideline given in the PEP 8 style guide. In many good text editors, there is a setting to display a subtle line that indicates where the 79 character limit is.

For more guidelines, check out the code layout section of PEP 8 in the notes below.

PEP 8 guidelines for code layout

Writing Modular Code

Tip:DRY (Don't Repeat Yourself)

  • Don't repeat yourself! Modularization allows you to reuse parts of your code. Generalize and consolidate repeated code in functions or loops. Tip:Abstract out logic to improve readability
  • Abstracting out code into a function not only makes it less repetitive, but also improves readability with descriptive function names. Although your code can become more readable when you abstract out logic into functions, it is possible to over-engineer this and have way too many modules, so use your judgement. Tip:Minimize the number of entities (functions, classes, modules, etc.)
  • There are tradeoffs to having function calls instead of inline logic. If you have broken up your code into an unnecessary amount of functions and modules, you'll have to jump around everywhere if you want to view the implementation details for something that may be too small to be worth it. Creating more modules doesn't necessarily result in effective modularization. Tip:Functions should do one thing
  • Each function you write should be focused on doing one thing. If a function is doing multiple things, it becomes more difficult to generalize and reuse. Generally, if there's an "and" in your function name, consider refactoring. Tip:Arbitrary variable names can be more effective in certain functions
  • Arbitrary variable names in general functions can actually make the code more readable. Tip:Try to use fewer than three arguments per function
  • Try to use no more than three arguments when possible. This is not a hard rule and there are times it is more appropriate to use many parameters. But in many cases, it's more effective to use fewer arguments. Remember we are modularizing to simplify our code and make it more efficient to work with. If your function has a lot of parameters, you may want to rethink how you are splitting this up.

Refactor: Wine Quality Analysis

In this exercise, you'll refactor code that analyzes a wine quality dataset taken from the UCI Machine Learning Repository here. Each row contains data on a wine sample, including several physicochemical properties gathered from tests, as well as a quality rating evaluated by wine experts.

The code in this notebook first renames the columns of the dataset and then calculates some statistics on how some features may be related to quality ratings. Can you refactor this code to make it more clean and modular?

import pandas as pd
df = pd.read_csv('./dataset/winequality-red.csv', sep=';')
df.head()
fixed acidity volatile acidity citric acid residual sugar chlorides free sulfur dioxide total sulfur dioxide density pH sulphates alcohol quality
0 7.4 0.70 0.00 1.9 0.076 11.0 34.0 0.9978 3.51 0.56 9.4 5
1 7.8 0.88 0.00 2.6 0.098 25.0 67.0 0.9968 3.20 0.68 9.8 5
2 7.8 0.76 0.04 2.3 0.092 15.0 54.0 0.9970 3.26 0.65 9.8 5
3 11.2 0.28 0.56 1.9 0.075 17.0 60.0 0.9980 3.16 0.58 9.8 6
4 7.4 0.70 0.00 1.9 0.076 11.0 34.0 0.9978 3.51 0.56 9.4 5

Renaming Columns

You want to replace the spaces in the column labels with underscores to be able to reference columns with dot notation. Here's one way you could've done it.

new_df = df.rename(columns={'fixed acidity': 'fixed_acidity',
                             'volatile acidity': 'volatile_acidity',
                             'citric acid': 'citric_acid',
                             'residual sugar': 'residual_sugar',
                             'free sulfur dioxide': 'free_sulfur_dioxide',
                             'total sulfur dioxide': 'total_sulfur_dioxide'
                            })
new_df.head() 
fixed_acidity volatile_acidity citric_acid residual_sugar chlorides free_sulfur_dioxide total_sulfur_dioxide density pH sulphates alcohol quality
0 7.4 0.70 0.00 1.9 0.076 11.0 34.0 0.9978 3.51 0.56 9.4 5
1 7.8 0.88 0.00 2.6 0.098 25.0 67.0 0.9968 3.20 0.68 9.8 5
2 7.8 0.76 0.04 2.3 0.092 15.0 54.0 0.9970 3.26 0.65 9.8 5
3 11.2 0.28 0.56 1.9 0.075 17.0 60.0 0.9980 3.16 0.58 9.8 6
4 7.4 0.70 0.00 1.9 0.076 11.0 34.0 0.9978 3.51 0.56 9.4 5

And here's a slightly better way you could do it. You can avoid making naming errors due to typos caused by manual typing. However, this looks a little repetitive. Can you make it better?

labels = list(df.columns)
labels[0] = labels[0].replace(' ', '_')
labels[1] = labels[1].replace(' ', '_')
labels[2] = labels[2].replace(' ', '_')
labels[3] = labels[3].replace(' ', '_')
labels[5] = labels[5].replace(' ', '_')
labels[6] = labels[6].replace(' ', '_')
df.columns = labels

df.head()
def replace_labels(columns):
    labels = list(columns)
    
    for i in range(len(labels)):
        labels[i] = labels[i].replace(' ', '_')
    return labels

df.columns = replace_labels(df.columns)

df.head()
fixed_acidity volatile_acidity citric_acid residual_sugar chlorides free_sulfur_dioxide total_sulfur_dioxide density pH sulphates alcohol quality
0 7.4 0.70 0.00 1.9 0.076 11.0 34.0 0.9978 3.51 0.56 9.4 5
1 7.8 0.88 0.00 2.6 0.098 25.0 67.0 0.9968 3.20 0.68 9.8 5
2 7.8 0.76 0.04 2.3 0.092 15.0 54.0 0.9970 3.26 0.65 9.8 5
3 11.2 0.28 0.56 1.9 0.075 17.0 60.0 0.9980 3.16 0.58 9.8 6
4 7.4 0.70 0.00 1.9 0.076 11.0 34.0 0.9978 3.51 0.56 9.4 5

Analyzing Features

Now that your columns are ready, you want to see how different features of this dataset relate to the quality rating of the wine. A very simple way you could do this is by observing the mean quality rating for the top and bottom half of each feature. The code below does this for four features. It looks pretty repetitive right now. Can you make this more concise?

You might challenge yourself to figure out how to make this code more efficient! But you don't need to worry too much about efficiency right now - we will cover that more in the next section.

df = pd.read_csv('./dataset/winequality-red.csv', sep=';')
median_alcohol = df.alcohol.median()
for i, alcohol in enumerate(df.alcohol):
    if alcohol >= median_alcohol:
        df.loc[i, 'alcohol'] = 'high'
    else:
        df.loc[i, 'alcohol'] = 'low'
print(df.groupby('alcohol').quality.mean())

median_pH = df.pH.median()
for i, pH in enumerate(df.pH):
    if pH >= median_pH:
        df.loc[i, 'pH'] = 'high'
    else:
        df.loc[i, 'pH'] = 'low'
print(df.groupby('pH').quality.mean())

median_sugar = df['residual sugar'].median()
for i, sugar in enumerate(df['residual sugar']):
    if sugar >= median_sugar:
        df.loc[i, 'residual sugar'] = 'high'
    else:
        df.loc[i, 'residual sugar'] = 'low'
print(df.groupby('residual sugar').quality.mean())

median_citric_acid = df['citric acid'].median()
for i, citric_acid in enumerate(df['citric acid']):
    if citric_acid >= median_citric_acid:
        df.loc[i, 'citric acid'] = 'high'
    else:
        df.loc[i, 'citric acid'] = 'low'
print(df.groupby('citric acid').quality.mean())
def wine_quality(column):
    median = df[column].median()
    for i, col in enumerate(df[column]):
        if col >= median:
            df.loc[i, column] = 'high'
        else:
            df.loc[i, column] = 'low'
            
    return df.groupby(column).quality.mean()
features = ['alcohol', 'pH', 'residual_sugar', 'citric_acid']

for feature in features:
    print(wine_quality(feature))
alcohol
high    5.958904
low     5.310302
Name: quality, dtype: float64
pH
high    5.598039
low     5.675607
Name: quality, dtype: float64
residual_sugar
high    5.665880
low     5.602394
Name: quality, dtype: float64
citric_acid
high    5.822360
low     5.447103
Name: quality, dtype: float64