Definitions
The import Statement
The as keyword
The dir() Function
Import from Module
Standard Library
OS
Dates
Math
RegEx
csv
PIP
Summary
A script is a Python file that’s intended to be run directly. When you run it, it should do something. This means that scripts will often contain code written outside the scope of any classes or functions.
A module is a Python file that’s intended to be imported into scripts or other modules. It often defines members like classes, functions, and variables intended to be used in other files that import it.
A package is a collection of related modules that work together to provide certain functionality. These modules are contained within a folder and can be imported just like any other modules. This folder will often contain a special init file that tells Python it’s a package, potentially containing more modules nested within subfolders
A library is an umbrella term that loosely means “a bundle of code.” These can have tens or even hundreds of individual modules that can provide a wide range of functionality. Matplotlib is a plotting library. The Python Standard Library contains hundreds of modules for performing common tasks, like sending emails or reading JSON data.
A Python library is a collection of related modules. It contains bundles of code that can be used repeatedly in different programs. It makes Python Programming simpler and convenient for the programmer. As we don’t need to write the same code again and again for different programs.
What’s special about the Standard Library is that it comes bundled with your installation of Python, so you can use its modules without having to download them from anywhere.
Every Python file is a module. If you create a file with .py extension that has some actual Python code in it, you’ve created a Python module.
Python comes included with a built-in standard library. This library is a set of script modules that may be accessed by a Python program to make programming easier and eliminate the need to rewrite commonly used commands.
A module is a file containing a set of functions you want to include in your application.
To create a module just save the code you want in a file with the file extension .py
You can name the module file whatever you like, but it must have the file extension .py
# example 1
# mymodule.py
def get_index(haystack, needle):
for index,item in enumerate(haystack):
if item == 'needle':
return index
else:
return -1
We can use the module we just created, by using the import statement
When using a function from a module, use the syntax: module_name.function_name
The module can contain functions and variables of all types (lists, dictionaries, objects etc)
# example 2
# new_file.py
import mymodule
colors = ["Red", "Blue", "Yellow", "Green", "Black"]
# we want to know the index of color "Yellow" in the list
mymodule.get_index(colors, "Yellow") # call the function
Let's add a variable to our mymodule.py file
# example 3
# mymodule.py
# define a variable
person = {"name":"John","age":60,"city":"Tokyo"}
# define a function
def get_index(haystack, needle):
"""function used to check if item exists in an iterable
if item is in the iterable return it's index.
it item is not in the iterable return -1
"""
for index,item in enumerate(haystack):
if item == 'needle':
return index
else:
return -1
Import the module named mymodule, and access the person dictionary
# example 4
# new_file.py
import mymodule
name = mymodule.person['name']
print(name)
as keyword¶You can create an alias (rename a module) when you import a module, by using the as keyword.
This is a way to use a short form of the module name. You can associate a suitable smaller name for a module when importing it.
Rename mymodule to mm
# example 5
# new_file.py
import mymodule as mm
name = mm.person['name']
print(name)
dir() Function¶The built-in function dir() returns a list of defined names in a namespace.
dir()
['In', 'Out', '_', '__', '___', '__builtin__', '__builtins__', '__doc__', '__loader__', '__name__', '__package__', '__spec__', '_dh', '_i', '_i1', '_i2', '_i3', '_ih', '_ii', '_iii', '_oh', 'exit', 'get_index', 'get_ipython', 'quit']
When given an argument that is the name of a module, dir() lists the names defined in the module
import math
dir(math)
['__doc__', '__loader__', '__name__', '__package__', '__spec__', 'acos', 'acosh', 'asin', 'asinh', 'atan', 'atan2', 'atanh', 'ceil', 'comb', 'copysign', 'cos', 'cosh', 'degrees', 'dist', 'e', 'erf', 'erfc', 'exp', 'expm1', 'fabs', 'factorial', 'floor', 'fmod', 'frexp', 'fsum', 'gamma', 'gcd', 'hypot', 'inf', 'isclose', 'isfinite', 'isinf', 'isnan', 'isqrt', 'ldexp', 'lgamma', 'log', 'log10', 'log1p', 'log2', 'modf', 'nan', 'perm', 'pi', 'pow', 'prod', 'radians', 'remainder', 'sin', 'sinh', 'sqrt', 'tan', 'tanh', 'tau', 'trunc']
from import <name(s)>An alternate form of the import statement allows individual objects from the module to be imported directly
frommodule_nameimportname(s)
<name(s)> can be called without the module name
# example 6
# mymodule.py
# define a variable
person = {"name":"John","age":60,"city":"Tokyo"}
# define a function
def get_index(haystack, needle):
"""function used to check if item exists in an iterable
if item is in the iterable return it's index.
it item is not in the iterable return -1
"""
for index,item in enumerate(haystack):
if item == 'needle':
return index
else:
return -1
# example 7
# new_file.py
# import just the get_index function
from mymodule import get_index
fruits = ["Mango","Apple","Orange","Pineapple"]
print(get_index(fruits, "Orange"))
# notice how we did not use mymodule.get_index()
# example 8
# using alias
from mymodule import get_index as gi
fruits = ["Mango","Apple","Orange","Pineapple"]
print(gi(fruits, "Orange"))
Now that we have learnt the import system, it's time we explore the Standard Library
The Python Standard Library contains a wide range of modules to deal with everyday programming and is included with the standard version of Python, meaning no additional installation is required. It provides modules for such tasks as interacting with the operating system, reading and writing CSV files, generating random numbers, and working with dates and time.
Let's start with the os module
os module¶The os module provides functions for interacting with the operating system. It contains numerous tools for working with directories, paths, and files.
To get the current working directory:
import os
# get the path of the current working directory
work_dir = os.getcwd()
print(work_dir)
C:\Users\Just Nick\Desktop\Analysis\Denaco\Python
os.chdir() function modifies the current working directory to the given path, returning None
# change the current working directory
os.chdir('C:/Users/Just Nick/Desktop')
# check if it worked
print(os.getcwd())
C:\Users\Just Nick\Desktop
os.listdir() returns a list of all files and directories in the specified path. If path is omitted, it returns a list of entries in the current working directory.
os.chdir('C:/Users/Just Nick/Desktop/Analysis/Denaco')
os.listdir()
['.ipynb_checkpoints', 'Advanced Excel.docx', 'CensusData', 'Instructor-resources.txt', 'misc', 'Python', 'R', 'SQL', 'stack-overflow-developer-survey-2019.zip', 'train.csv']
We can use the os.listdir() function in combination with other functions to filter the returned list. As shown below, we use the string method .endswith() to obtain a list of python files present in the current working directory.
# example 9
os.chdir('C:/Users/Just Nick/Desktop/Analysis/Denaco/Python')
# all entries
os.listdir()
['.ipynb_checkpoints', 'bmi-solution.ipynb', 'comprehensive-guide-to-pandas.ipynb', 'conditional expressions in python.ipynb', 'denaco-manu-python-curriculum - Sheet1.pdf', 'fizz_buzz.py', 'functions.ipynb', 'Intro to python', 'loops and iterations in python.ipynb', 'mastery_of_pandas.ipynb', 'modules and standard library.ipynb', 'Pandas-Demo', 'python-resources.txt', 'Python.docx', 'string-formating.py', 'train.csv']
# example 9 continued
os.chdir('C:/Users/Just Nick/Desktop/Analysis/Denaco/Python')
# all entries
entries = os.listdir()
# get only .py files
py_files = [entry for entry in entries if entry.endswith('.py')]
print(py_files)
['fizz_buzz.py', 'string-formating.py']
Python os module allows you to rename a file or directory programmatically using the os.rename(__src__,__dst__) function. This function renames the file or directory src to dst and returns None.
If you try to rename a nonexistent file or directory Python raises an OSError exception.
Let's rename the file 'string-formating.py' to 'python-string-formating.py'
# example 10
os.chdir('C:/Users/Just Nick/Desktop/Analysis/Denaco/Python')
# rename string-formating.py to python-string-formating.py
os.rename('string-formating.py','python-string-formating.py')
os.listdir()
['.ipynb_checkpoints', 'bmi-solution.ipynb', 'comprehensive-guide-to-pandas.ipynb', 'conditional expressions in python.ipynb', 'denaco-manu-python-curriculum - Sheet1.pdf', 'fizz_buzz.py', 'functions.ipynb', 'Intro to python', 'loops and iterations in python.ipynb', 'mastery_of_pandas.ipynb', 'modules and standard library.ipynb', 'Pandas-Demo', 'python-resources.txt', 'python-string-formating.py', 'Python.docx', 'train.csv']
The function os.mkdir(path) allows you to create a directory named path
Let's create a new directory called our_new_dir in our working directory.
# example 11
os.chdir('C:/Users/Just Nick/Desktop/Analysis/Denaco/Python')
entries = os.listdir()
print(entries)
# create a new directory names
os.mkdir('our_new_dir')
# check the modification
print(os.listdir())
['.ipynb_checkpoints', 'bmi-solution.ipynb', 'comprehensive-guide-to-pandas.ipynb', 'conditional expressions in python.ipynb', 'denaco-manu-python-curriculum - Sheet1.pdf', 'fizz_buzz.py', 'functions.ipynb', 'Intro to python', 'loops and iterations in python.ipynb', 'mastery_of_pandas.ipynb', 'modules and standard library.ipynb', 'Pandas-Demo', 'python-resources.txt', 'python-string-formating.py', 'Python.docx', 'train.csv'] ['.ipynb_checkpoints', 'bmi-solution.ipynb', 'comprehensive-guide-to-pandas.ipynb', 'conditional expressions in python.ipynb', 'denaco-manu-python-curriculum - Sheet1.pdf', 'fizz_buzz.py', 'functions.ipynb', 'Intro to python', 'loops and iterations in python.ipynb', 'mastery_of_pandas.ipynb', 'modules and standard library.ipynb', 'our_new_dir', 'Pandas-Demo', 'python-resources.txt', 'python-string-formating.py', 'Python.docx', 'train.csv']
The function os.mkdir() creates a directory in an existing directory. If you try to create a directory in a place that does not exist, an exception is raised. Alternatively, you can use the function os.mkdirs() (with an s) that creates intermediate folders if they do not exist.
These are only a few of the functions available in the os module.
datetime module¶The datetime library provides many tools for working with dates and times in Python. We can easily get the current time, subtract two dates, or convert dates into custom-formatted strings using the datetime module.
# example 12
import datetime
# get today's date
today = datetime.date.today()
print(today)
2022-03-09
# example 13
# create a datetime object - George Orwell birth date
birth = datetime.datetime(1903, 6, 25)
print(type(birth))
# <class 'datetime.datetime'>
# access attribute - dot notation
y = birth.year
# 1903
m = birth.month
print("Year", y)
print("month:", m)
<class 'datetime.datetime'> Year 1903 month: 6
# get current date and time
now = datetime.datetime.now()
print("Type:",type(now))
print(now)
Type: <class 'datetime.datetime'> 2022-03-09 10:55:11.387432
the function returns a datetime object with the following format: YYYY-MM-DD HH:MM:SS:MS.
We can subtract two datetime objects in Python, obtaining as a result a timedelta object. This object represents the time span between both dates.
# example 14
# create a datetime object - George Orwell birth date
birth = datetime.datetime(1903, 6, 25)
# create a datetime object - George Orwell death date
death = datetime.datetime(1950, 1, 21)
# calculate how long he lived by subtracting both dates
live = death - birth
print(live)
17012 days, 0:00:00
math module, that allows you to perform mathematical tasks on numbers.
math.sqrt() method returns the square root of a number
# example 15
import math
x = math.sqrt(64)
print(x)
8.0
The math.ceil() method rounds a number upwards to its nearest integer, and the math.floor() method rounds a number downwards to its nearest integer, and returns the result
# example 16
x = math.ceil(1.4)
y = math.floor(1.4)
print("Ceiling", x)
print("Floor", y)
Ceiling 2 Floor 1
The math.factorial() method returns the factorial of a number.
# example 17
y = math.factorial(3) # 3 * 2 * 1
print(y)
6
re module¶A RegEx, or Regular Expression, is a sequence of characters that forms a search pattern.
A Regular Expressions (RegEx) is a special sequence of characters that uses a search pattern to find a string or set of strings in a piece of text or even numerical data. It can detect the presence or absence of a text by matching it with a particular pattern, and also can split a pattern into one or more sub-patterns.
RegEx can be used to check if a string contains the specified search pattern.
Python has a built-in package called re, which can be used to work with Regular Expressions.
re.findall() returns a list containing all matches
# example 18
import re
txt = "The rain in Spain"
x = re.findall('ai', txt)
print(x)
['ai', 'ai']
re.sub() replaces the matches with the text of your choice
Example: replace all the space characters with dashes
# example 19
txt = "The rain in Spain"
x = re.sub('\s','-',txt)
print(x)
The-rain-in-Spain
Regular expressions use the backslash character ('\') to indicate special forms or to allow special characters to be used without invoking their special meaning.
csv module¶A comma-separated values (CSV) file is a common format used to transfer information.
The information is structured as a table where each row contains a record and each column a field, being the fields separated by commas. Although commas are the most common separator, we can use other delimiters such as spaces or tabs.
To write data to a CSV file, first, we open the CSV file in writing mode with the built-in function open. Then, we provide the open file object returned by the open function as input to the csv.writer() function, obtaining a writer object.
The writer object supports two methods for writing data to a CSV file:
csvwriter.writerow(row) -> This method writes a row of data to the csv file
csvwriter.writerows(rows) -> This method writes all given rows to the csv file
# example 20
fields = ['id', 'name', 'email', 'age']
# create some users
john = {
'id':1,
'name':"John Doe",
'email':'john@email.com',
'age':45
}
jane = {
'id':2,
'name':"jane Doe",
'email':'jane@email.com',
'age':25
}
kev = {
'id':3,
'name':"Kev Joe",
'email':'kev@gmail.com',
'age':35
}
# create a list of users
users = [john,jane,kev]
users
[{'id': 1, 'name': 'John Doe', 'email': 'john@email.com', 'age': 45},
{'id': 2, 'name': 'jane Doe', 'email': 'jane@email.com', 'age': 25},
{'id': 3, 'name': 'Kev Joe', 'email': 'kev@gmail.com', 'age': 35}]
# example 20 continuation
import csv
# create a new csv file called users.csv
f = open('users.csv', 'w')
# create a writer object
DictWriter = csv.DictWriter(f, fieldnames=fields)
# write header
DictWriter.writeheader()
# use the writerows method to insert user data into csv file
DictWriter.writerows(users)
# close the file
f.close()
entries = os.listdir()
print(entries)
['.ipynb_checkpoints', 'bmi-solution.ipynb', 'comprehensive-guide-to-pandas.ipynb', 'conditional expressions in python.ipynb', 'denaco-manu-python-curriculum - Sheet1.pdf', 'fizz_buzz.py', 'functions.ipynb', 'Intro to python', 'loops and iterations in python.ipynb', 'mastery_of_pandas.ipynb', 'modules and standard library.ipynb', 'our_new_dir', 'Pandas-Demo', 'python-resources.txt', 'python-string-formating.py', 'Python.docx', 'train.csv', 'users.csv']
PIP is a package manager for Python packages, or modules if you like.
With PIP downloading a package is very easy.
Open the command line interface and tell PIP to download the package you want.
pip install mysql-connector-python
Once the package is installed, it is ready to use. Import the package into your project
import mysql-connector
mydb = mysql.connector.connect(
host="localhost",
user="yourusername",
password="yourpassword"
)
print(mydb)
pip list
Package VersionNote: you may need to restart the kernel to use updated packages. -------------------- --------- anyio 3.5.0
WARNING: You are using pip version 21.1.1; however, version 22.0.4 is available. You should consider upgrading via the 'c:\users\just nick\appdata\local\programs\python\python38\python.exe -m pip install --upgrade pip' command.
argon2-cffi 21.3.0 argon2-cffi-bindings 21.2.0 asttokens 2.0.5 attrs 21.4.0 Babel 2.9.1 backcall 0.2.0 beautifulsoup4 4.10.0 black 22.1.0 bleach 4.1.0 certifi 2021.10.8 cffi 1.15.0 charset-normalizer 2.0.10 click 8.0.3 colorama 0.4.4 cycler 0.11.0 debugpy 1.5.1 decorator 5.1.1 defusedxml 0.7.1 entrypoints 0.3 et-xmlfile 1.1.0 executing 0.8.2 fake-useragent 0.1.11 fonttools 4.29.1 idna 3.3 importlib-resources 5.4.0 ipykernel 6.7.0 ipython 8.0.1 ipython-genutils 0.2.0 jedi 0.18.1 Jinja2 3.0.3 joblib 1.1.0 json5 0.9.6 jsonschema 4.4.0 jupyter-client 7.1.2 jupyter-core 4.9.1 jupyter-server 1.13.4 jupyterlab 3.2.8 jupyterlab-pygments 0.1.2 jupyterlab-server 2.10.3 kiwisolver 1.3.2 MarkupSafe 2.0.1 matplotlib 3.5.1 matplotlib-inline 0.1.3 mistune 0.8.4 mypy-extensions 0.4.3 nbclassic 0.3.5 nbclient 0.5.10 nbconvert 6.4.1 nbformat 5.1.3 nest-asyncio 1.5.4 notebook 6.4.8 numpy 1.22.1 openpyxl 3.0.9 packaging 21.3 pandas 1.4.0 pandocfilters 1.5.0 parso 0.8.3 pathspec 0.9.0 pickleshare 0.7.5 Pillow 9.0.0 pip 21.1.1 platformdirs 2.4.1 prometheus-client 0.13.1 prompt-toolkit 3.0.26 psycopg2 2.9.3 pure-eval 0.2.2 pycparser 2.21 Pygments 2.11.2 pyparsing 3.0.7 pyrsistent 0.18.1 python-dateutil 2.8.2 pytz 2021.3 pywin32 303 pywinpty 2.0.1 pyzmq 22.3.0 requests 2.27.1 scikit-learn 1.0.2 scipy 1.7.3 seaborn 0.11.2 Send2Trash 1.8.0 setuptools 56.0.0 six 1.16.0 sniffio 1.2.0 soupsieve 2.3.1 stack-data 0.1.4 terminado 0.13.1 testpath 0.5.0 threadpoolctl 3.1.0 tomli 2.0.0 tornado 6.1 traitlets 5.1.1 typing-extensions 4.0.1 urllib3 1.26.8 wcwidth 0.2.5 webencodings 0.5.1 websocket-client 1.2.3 xlrd 2.0.1 zipp 3.7.0
Their name (required, must have at least two characters)
email (has to be a valid email, with @ and . and extension)
phone number (ensure its valid, if it starts with +254 it should have 12 digits, if it starts with 07 then it should be 10 digits)
A strong password (must be at least 8 characters, must have a digit, at least one lowercase letter, at least one uppercase letter, must have at least one special character (?;@!=/()-.,), must not have a space character)
Their age
After ensuring that the user has provided the correct details, add the user details to a csv file called 'user_details.csv'
The csv file should have 6 fields.
Name
phone
password
age
category (if age < 18, should be Minor if age is between 18 and 50 this field should have Adult. If age is above 50 years indicate they are Senior)
date when the user was added to the file (should not come from input)
Add at least 5 users to this csv file.
### Challenge 1 solution
import re
import csv
import os
import datetime
def get_name():
"""
get the users name.
verify if it's correct
return string name
"""
name = input("Enter your name: ")
# remove any whitespace from input (space at the beginning or end)
name = name.strip()
if name != "" and len(name) >= 2:
return name
else:
get_name()
def get_email():
"""
get the user's email.
verify it has an '@' a '.' and an extension
if verified return a string with email
"""
email = input("Enter your email: ")
email = email.strip() # remove any whitespace
if email == "":
# if email was not provided, ask the user for email again
return get_email()
# define a regular expression to verify it's a valid email address
# example email : example@email.com
# The search() function searches the string for a match, and returns a Match object if there is a match.
# If no matches are found, the value None is returned
match = re.search('\w+@\w+\.\w+',email) # returns a match object
if match is None:
return get_email()
else:
return email # return a string
def get_phone():
"""
get the user's phone.
can be either 2547xxxxxxxx or 07xxxxxxxx
if user inputs 07xxxxxxxx change it to use country code format 2547xxxxxxxx
verify it's correct format and all digits
if it's correct format return the phone number as a string
"""
phone = input("Enter your phone: ")
# remove white space
phone = phone.strip()
# remove any space
phone = phone.replace(" ","")
# validate phone
# check if user's input started with a + sign
if phone.startswith('+'):
phone = phone.replace("+","")
# if phone input begins with a 0 ensure it's 10 digits long
if len(phone) == 10:
# ensure it's all digits
try:
map(int, phone) # check thal all elements can be cast to int
except:
print("use integers only")
return get_phone()
# now change it to use country code format
phone = phone.replace('0', '254')
elif len(phone) == 12:
# ensure it's all digits
if not phone.isnumeric():
print("phone can only have numeric characters")
return get_phone()
else:
print("length of phone is invalid")
return get_phone()
return phone
def get_password():
"""
get user's password
"""
password = input("Enter your password (must be at least 8 characters, must have a digit, \
at least one lowercase letter, \
at least one uppercase letter, \
must have at least one special character (?;@!=/()-.,) \
must not have a space character): ")
# remove any whitespace from input
password = password.strip()
# ensure password has at least 8 characters
if len(password) < 8:
print("password must be at least 8 characters long")
return get_password()
# check if the email has a space
# The find() method returns -1 if the value is not found.
if password.find(" ") != -1:
print("password should not have a space")
return get_password()
# check if password has at least one special character (?;@!=/()-.,)
match = re.search('[?;@!=/()-.,]',password)
if match is None:
print("Password must have at least one special characters (?;@!=/()-.,)")
return get_password()
# to check if there is a lowercase and uppercase letters, use a loop
upper = 0
lower = 0
for char in password:
# if we already have at least one lower and at least one upper case, then stop looping
if upper > 0 and lower > 0:
break
# ensure that the character is an alphabet
if char.isalpha():
if char.isupper():
upper += 1
else:
lower += 1
if lower > 0 and upper > 0:
return password
return get_password()
def get_age():
age = input("Enter your age: ")
# remove any whitespace character
age = age.strip()
try:
age = int(age)
except:
print("use digits for age")
return get_age()
return age
def add_user(user_dict):
"""
accepts a dictionary of user details as input
if user input all the details in the correct format add the user to the user_details csv_file
returns True if user was added to file
False if we had a problem adding user to file
"""
# check if user_details.csv already exists
path = 'user_details.csv'
if not os.path.exists(path):
# the path does not exist so create the file
f = open(path, 'w') # open in write mode
fields = ['name','email','phone','password','age','category','date']
# create a writer object
DictWriter = csv.DictWriter(f, fieldnames=fields)
# write header
DictWriter.writeheader()
# use the writerow method to insert the user's data into csv file
DictWriter.writerow(user_dict)
# close the file
f.close()
else:
# the csv file already exists so we need to append to the file
with open('user_details.csv','a') as f:
# create a writer object
DictWriter = csv.DictWriter(f, fieldnames=fields)
# use the writerow method to insert the user's data into csv file
DictWriter.writerow(user_dict)
return True
def get_details():
"""
ask the user for details from input one by one and verify if it's valid according to our policy.
if a user inputs details that do not meet our requirements, prompt the user to enter the details for that field only
"""
user = {} # initialize a dictionary of user details
# get user details from relevant functions
name = get_name()
email = get_email()
phone = get_phone()
password = get_password()
age = get_age()
if age < 18:
category = 'Minor'
elif 18 <= age < 50:
category = 'Adult'
else:
category = 'Senior'
user['name'] = name
user['email'] = email
user['phone'] = phone
user['password'] = password
user['age'] = age
user['category'] = category
user['date'] = str(datetime.date.today())
print(user)
return add_user(user)
def get_users():
"""
use this function to retrieve a list of all users in our file.
returns a list of dictionaries of user details
"""
if not os.path.exists('user_details.csv'):
print("No Users Yet")
return None
else:
# read the csv file
with open('user_details.csv', 'r') as csv_file: # open in read mode
csv_reader = csv.DictReader(csv_file)
# initialize list for holding user data
users = []
for row in csv_reader:
print(f"Name: {row['name']} email: {row['email']} age: {row['age']}")
users.append(row)
return users
user_1 = get_details()
print(user_1)
the user's city
the user's continent (think of how you can validate)
Use the getpass module to prompt the user for their password (more secure)
Write the details in the csv file. Add the following fields:
city -> cannot be null
continent -> has to be a valid continent
Time: Time in hours:seconds when user was added to file
### Your solution here
##