python 3 programming

Python 3 Programming

Why Python? Python is very beginner-friendly. The syntax (words and structure) is extremely simple to read and

follow, most of which can be understood even if you do not know any programming. Let me show you:

Garage = "Ferrari", "Honda", "Porsche", "Toyota"

each_car Garage: for in

(each_car) print

"print()" is a built-in Python function that will output some text to the console.

Looking at the code about cars in the garage, can you guess what will happen? You probably have a

general idea. For each_car in the garage, we're going to do something. What are we doing? We are

printing each car.

What can Python do?

Python is a fully-functional programming language that can do anything almost any other language can

do, at comparable speeds.

Python is capable of threading and GPU processing just like any other language. Most of the data

processing modules are actually just Python wrappers around C/C++ code.

"Modules" are pre-written Python code that you "import" in your Python program. Since there are many

tasks that people commonly do, we have modules that people have written that do these task for you,

and they usually do them in the cleanest and most efficient method possible. Sometimes you will see

people refer to "DRY." This stands for Don't Repeat Yourself, which often also translates into "Don't

Repeat Someone Else."

The phrase "wrapper" means that someone has placed, like a wrapper, Python code over another

language. So, when you have a Python wrapper around C++ code, what someone has done is written

some Python code that interacts with the C++ language. This allows you to make use of various aspects

of the language being wrapped, in this case C++, without actually needing to know or understand that

language.

Thus, Python can be used to make games, do data analysis, control robot and hardware, create GUIs, or

even to create websites.

"GUI" stands for Graphical User Interface, and is used to describe a program that incorporates graphics

to make the program more interactive for the user.

Print Function and Strings

The print function in Python is a function that outputs to your console window whatever you say you

want to print out. At first blush, it might appear that the print function is rather useless for

programming, but it is actually one of the most widely used functions in all of python. The reason for

this is that it makes for a great debugging tool.

"Debugging" is the term given to the act of finding, removing, and fixing errors and mistakes within

code.

If something isn't acting right, you can use the print function to print out what is happening in the

program. Many times, you expect a certain variable to be one thing, but you cannot see what the

program sees. If you print out the variable, you might see that what you thought was, was not.

Next up, strings, what are they? Strings are just "strings" of text, hence the name. Strings are a type of

data. Another type of data is integers.

print('Single Quotes')

print("double quotes")

We're printing out a string. Notice that the quotes are single quotes. You can use single quotes or

double quotes, but they need to be used together.

While we're talking about strings and the print function, it would be useful to discuss concatenation.

Concatenation just means the combination of things. You can use the "+" or the "," to join strings

together. If you use a ",", then you will have a space in between the strings you joined. If you use a "+",

then the strings will be strung together with no space. You will need to add one if you wanted.

If you use the "+" to join integers and floats together, then you will perform an arithmetic operation. If

you use the ",", then it will print them out separately, with a space.

print('can do this',5)

print('cannot do this:'+5)

print(‘5’,’+5.5555555555’)

It is also important to bring up how to put quotes within strings. You can either put double quotes inside

single quotes, singles inside doubles, or use the "\" backslash. The \ character is known as an escape

character, and it will "escape" the characteristic of the following character and just take on the 'visual'

aspect of it.

The purpose of the "escape character" is to escape various characteristics for characters. For example, a

quotation, ", in a string might wreak havoc. Take for example: x = "He said, "Hello there!" "

Yeah, that's going to be a problem. There are obviously many options to avoid this specific problem, but

one of them would be to use an escape character:

x = "He said, \"Hello there!\" "

If you do a print(x), you will not see the escape characters, and you will see the quotes. Sometimes you

want to show the escape character as well:

x = "An escape character is a \"

How might you solve that?

Here are some examples of quotation rules:

print('Can't do this')

print('you\'ll have success here')

print("you'll have success here too")

It is also important to bring up how to put quotes within strings. You can either put double quotes inside

single quotes, singles inside doubles, or use the "\" backslash. The \ character is known as an "escape"

character, and it will "escape" the characteristic of the following character and just take on the 'visual'

aspect of it.

Comments

# this is a single line comment

You can also do multi-line comments like:

'''

Multiple line comment, can use triple "double quotes" as well to do

this.

'''

print('''

This is a massive print, where you want to

use multiple lines, maybe make designs, or

something like that.

''')

Getting User Input For a simple text-based GUI (graphical user interface), it can sometimes be useful to allow for a user to

enter some input into the program while it runs. Using Python 3's "input" function, we can do that.

In time, you may want to eventually learn how to make GUIs in windows, but you will still find yourself

needing raw input from time to time, like text fields, even in these sorts of GUIs.

x = input('What is your name?: ')

print('Hello',x)

If you enter your name, the output will be a hello and your name.

Calculator: In Python 3 doing math is very well works and extremely simple. As like using simple calculator.

Variables In almost every single Python program you write, you will have variables. Variables act as placeholders

for data. They can aid in short hand, as well as with logic, as variables can change, hence their name.

Variables help programs become much more dynamic, and allow a program to always reference a value

in one spot, rather than the programmer needing to repeatedly type it out, and, worse, changes it if

they decide to use a different definition for it.

Variables can be called just about whatever you want. You wouldn't want them to conflict with function

names, and they also cannot start with a number.

You want to be careful what you name variables, classes (discussed later), and functions (discussed

later), so that they do not have the same names as each other.

For example, you have learned about the print function. What if you go and define a variable named

print?

Say, for example, you do:

print = print("Uh oh!")

Now you have a variable and a function named print, which can cause trouble down the road!

NewVar = 91

print(NewVar)

In this case, we will have a 91 printed out to console. So, in this case, we were able to store an integer to

our variable.

cannotDo = Hey!

Hey! is not a valid datatype, and this will throw an error. You would need to throw quotes around the

string.

canDo = 'Hey!'

print(canDo)

canContainOperations = 5/4

print(canContainOperations)

Here, we can see that we were even able to store the result of a calculation to our variable.

We can even store a variable to our variable or an operation with our variables to a variable. Something

like var3 = (var2/var1) would work. You can store other things, like functions, as well to variables.

Loops and Statements In this section we are going to see about while, for loops and if statements. The two distinctive loops we

have in Python 3 logic are the "for loop" and the "while loop." Both of them achieve very similar results,

and can almost always be used interchangeably towards a goal. Many times it comes down to

programmer preference, or is reliant on efficiency. Generally, for loop can be more efficient than the

while loop, but not always.

While Loop

The idea of the While loop is:

While something is the case, do the following block of code.

Here is an example of a while loop:

condition = 1

while condition < 10:

print(condition)

condition += 1

Next, we specify the terms of the while statement, which are: While the condition variable is less than

10, we will print the condition variable out. After printing out the condition, we will add 1 to the current

condition. This process will continue until condition equals 10.

This setup of a while loop is known as creating a "counter," since basically that is what we're doing.

We're saying we just want to count 1 for every iteration and eventually stop at our limit. While loops are

usually finite and defined in this sense, but while loops can also be undefined.

For Loop

The next loop is the For loop. The idea of the for loop is to "iterate" through something. For each thing

in that something, it will do a block of code.

Typically, you will see the while loop being used for finite tasks that have predetermined length, and the

for loop being used for tasks that have uncertain and variable time-frames.

That said, the for loop can be used for the exact same tasks as the while loop.

Here's an example of a for loop:

exampleList = [1,5,6,6,2,1,5,2,1,4]

for x in exampleList:

print(x)

This code will print out each item in that list. Usually, people will choose to actually do something with the item in the list, more than just printing it out, but this is just a basic example.

Another example of a for loop is in the case of something like:

for x in range(1,11):

print(x)

This code is actually what is known as a generator function, and is highly efficient. The above works very

much like the "counter" function we made with a while loop. The only difference is this one is much

faster and more efficient in many cases.

IF Statement

The if statement is one of the most basic forms of logic that can be introduced into your program. The

idea of the if statement is to assess whether something is the case, and, if it is, then to perform the

following block of code within the statement.

Here is an example of an if statement:

x = 5

y = 10

if x > y:

print('x is greater than y')

Here, we define the variables x and y. Then, we ask if x is greater than y. If it is, then we will print 'x is greater than y.' If it is not, then nothing will happen.

In this case, 5 is not greater than 10, so nothing will happen.

x = 5

y = 10

if x < y:

print('x is less than y')

Now, we've flipped the comparison operator, to say if x is less than y. Basically, if 5 is less than 10. In this

case, the statement is true, and the output will be 'x is less than y.'

IF-Else Statement

The If-Else statement is designed to build on the if statement's logic. Here, we ask if something is the

case, and, if it is we do something. Then we say otherwise, which is contingent on the previous if

statement, do something else. If the previous if statement is true, then the else will not run. If the if

statement is false, then the else statement will run.

Here's an example:

x = 5

y = 8

if x > 55:

print('x is greater than 55')

else:

print('x is not less than 55')

It is important to note that the else statement only applies to the most recent if statement. You can

stack multiple if statements, but the else statement will only be contingent on the if statement that is

right above it.

IF Elif Else Statement

Now we bring the in "elif" statement. The elif allows us to tie multiple if statements together as we

might have intended to before with multiple if statements before we learned that the else will only be

contingent on the if statement above it.

The "elif" statement is a hybrid of the else and the if. The way it works is:

We ask if something is the case. If it is, then the elif and the else will not run. If it is not, then the elif will

run and question the if statement. If it is True, then it will run and the else will not run. If it is not true,

then the else statement will run. Here's an example:

x = 5

y = 10

z = 22

if x > y:


elif x < z:

print('x is less than z')

else:

print('if and elif never ran...')

Here, we ask if x is greater than y first. 5 is not greater than 10, so this is False. So the elif runs to ask if x

is less than z. In this case, it is asking if 5 is less than 22. It is, so we will see a print out saying x is less

than z. The "else" part of this will not run.

x = 5

y = 10

z = 22

if x > y:


elif x > z:

print('x is less than z')

else:

print('if and elif never ran...')

Here, we ask if x is greater than y first. 5 is not greater than 10, so this is False. So the elif runs to ask if x

is greater than z. In this case, it is asking if 5 is greater than 22. This is false, so it does not run. Then, we

find ourselves at the else statement, which notifies us that 'if and elif never ran...'

Functions The idea of a function is to assign a set of code, and possibly variables, known as parameters, to a single

bit of text. You can think of it a lot like why you choose to write and save a program, rather than writing

out the entire program every time you want to execute it.

To begin a function, the keyword 'def' is used to notify python of the impending function definition,

which is what def stands for. From there, you type out the name you want to call your function. It is

important to choose a unique name, and also one that won’t conflict with any other functions you might

be using. For example, you wouldn't want to go calling your function print.

def example():

print('this code will run')

z = 3 + 9

print(z)

Here we've called our function example. After the name of the function, you specify any parameters of

that function within the parenthesis parameters act as variables within the function, they are not

necessary to create a function, so first let's just do this without any parameters.

Now if we just run this, we see nothing happens. We have to actually call this function to execute,

because all we've done so far is just define the function and what it does. To run it, you can either type

out the function in the console like so:

example()

This is notably a very basic function. We can put all types of code into a function. We can put if

statements in there, run other functions in them, all sorts of things. Also, we can begin to learn how to

use function parameters and then later function parameter defaults.

Functional Parameters The idea of function parameters in Python is to allow a programmer who is using that function, define

variables dynamically within that function. For example:

def simple_addition(num1,num2):

answer = num1 + num2

print('num1 is', num1)

print(answer)

simple_addition(5,3)

Here, we defined our function name as simple_addition. In the function parameters (often called parms

for short), we specify variables named num1 and num2.

Next, within the function, we say this new answer variable is equal to whatever num1 plus num2 is. We

then print out what num1 is, whatever it happens to be. Finally, the last line of this function just prints

out the answer variable, which is num1 plus num2.

Now, to run this function and make use of these parameters, we run simple_addition(5,3). This runs the

simple_addition function using the parameters of num1=5 and num2=3. Then our program sums 5 and 3

together, then we print out that num1 is 5, and finally we print out the "answer" which was defined

already, which is the sum of 5 and 3, which is of course 8.

There is no limit to the amount of function parameters you have. If you want to just specify the

definitions of these parameters without saying the parameter, like when we just said 5 and 3, instead of

putting the parameter=5, then you must put them in the exact order. If you have a lot of parameters

where it might be difficult to remember their order, you could do something like:

simple_addition(num2=3,num1=5)

In that case, when you call the function and define the parameters, you can see how we actually defined num2 before num1, even though in the function definition we ask for them in the other way around. As long as you specify the parameter you are defining, you can jumble them up. Otherwise, you must keep them in order!

Finally, not only must they be in perfect order, but you must not specify too many or two few definitions.

This will not work:

simple_addition(3,5,6)

simple_addition(3)

Sometimes, it can make a lot of sense to make a complex function highly customize-able. That said,

some people may want to use the function in it's simple... "default" ... form. Think about it like buying a

car. Some people want to just buy the base model, with all the features that come by default from the

factory. Other people want to customize their car in many different ways.

Luckily, we allow people who want to customize their car the option to do it, but we do not ask every

buyer of every car what kind of wheels they want, what brand of tire, what screws they want, what kind

of leather seats by what brand, what windshield, what steering wheel, what lights... etc. This is just too

much for some people and what they want to use the car for. Same goes for functions in programming.

So we have function parameter defaults, which allow the function's creator to set "default" values to the

function parameters. This allows anyone to use a function with the default values, yet lets anyone who

wishes to customize them the ability to specify different values.

When using defaults, any parameters with defaults should be the last ones listed in the function's

parameters.

def simple(num1, num2=5):

pass

This is just a simple definition of a function, with num1 not being pre-defined (not given a default), and

num2 being given a default.

def basic_window(width,height,font='TNR'):

# let us just print out everything

print(width,height,font)

basic_window(350,500)

Here, we can see that, if there is a function parameter default, then, when we call that function, we do

not need to define or even mention that parameter at all!

basic_window(350,500,font='courier')

Here's another example, only this time we see that, despite the parameter having a default, we are able

to still change it.

Global and Local Variables In this section we're going to now discuss the concept of global and local variables. When users begin

using functions, they can quickly become confused when it comes to global and local variables... getting

a dreaded variable is not defined even when they clearly see that it is... or so they think.

These terms of global and local correspond to a variable's reach within a script or program. A global

variable is one that can be accessed anywhere. A local variable is the opposite; it can only be accessed

within its frame. The difference is that global variables can be accessed locally, but not modified locally

inherently. A local variable cannot be accessed globally, inherently. Now, don’t worry about committing

that to memory right now, I think it makes a lot more sense when you just see and do it, so let's do that.

# this variable has no parent function, but is actually NOT a global

variable.

# it just so happens that it is committed to memory before the function is

called

# so we are able to iterate, or call it out, but we cannot do much else.

x = 6

def example():

print(x)

# z, however, is a local variable.

z = 5

# this works

print(z)

example()

# this does not, which often confuses people, because z has been defined

# and successfully even was called... the problem is that it is a local

# variable only, and you are attempting to access it globally.

print(z)

Here, we can see that we are able to access the x variable. We then define and print out the z variable.

We can then call the function, and all seems well. When we go to reference the z variable however, we

have trouble. The z variable is local to the example function.

Let's look at another example:

x = 6

def example2():

# works

print(x)

print(x+5)

# but then what happens when we go to modify:

x+=6

# so there we attempted to take the x var and add 6 to it... but now

# we are told that we cannot, as we're referencing the variable before

# its assignment.

Here, again, we are able to reference x, we are even able to print x+6... but we are not allowed to

modify x.

What if we'd like to modify x? Well, then we need to use global!

x = 6

def example3():

# what we do here is defined x as a global variable.

global x

# now we can:

print(x)

x+=5

print(x)

Now we're cooking! The problem here is that some people do not like the idea at all of using global

variables. How do we get around using them and referencing them locally?

x = 6

def example4():

globx = x

# now we can:

print(globx)

globx+=5

print(globx)

We are able to do the above, by assigning the value that we can reference to a local variable, then doing

what we want with it from there.

Another choice you might have, as suggested by one of my viewers is the following:

x = 6

def example(x):

print(x)

x+=5

print(x)

return x

x = example(x)

print(x)

In the above example, we have the function modifying x. It may appear somewhat confusing since x is

being used in multiple locations, so maybe a clearer example is something like:

x = 6

def example(modify):

print(modify)

modify+=5

print(modify)

return modify

x = example(x)

print(x)

So, you can better visualize this function as a "modification" function, where it modifies the variable you

pass through. Besides the definition of this function, you only need to reassign the variable you want to

change as the function with that variable as the parameter.

Basic Error Debugging In this section we'll be discussing some of the basics to debugging. You get a lot of questions for help

where you have errors in code and are not sure what the problem is. If they used some extremely

simple debugging, they'd realize how obvious the answer is. Most of the time, the problem is a typo,

followed closely by a misunderstanding of indentation and standards.

Standards how are how you organize your code. With python, unlike most languages, you define blocks

of code, like functions, by indentation. Most python editors will automatically indent for you where it is

necessary. With this, if you are ever coding along and find python automatically indenting you where

you don't think it should, this should raise a flag for you to figure out.

The first error we'll discuss is the NameError: is not defined.

As obvious as this might appear to you, this gets people amazingly frequently. Just learn to recognize the

"is not defined" as a high chance that you have typoed the definition of the variable or when you are

calling it. Find what is wrong with this code block:

variable = 55

print(varaible)

Next up, we have indentation issues. You will see "expected an indented block" as a popup when you

never enter an indented block for something that requires it, like a function.

def task1():

def task2():

print('more tasks')

So there was a lacking expected indent. How about an unexpected one?

def task():

print ('stuff')

print('more stuff')

print('stuff')

Now what happens when you don't close off your strings and move to another line?

def task():

print('some people find themselves committing this too

print('ouch...')

Writing to a File It should be noted that there are two methods for saving data to a file, and those are writing and

appending. Writing to a file will write that bit of data, whatever it is, solely, to the file. This means if

there was anything there before, it will be gone if you use write.

If you use append, then you will basically add to whatever is previously there. I will be showing both

methods, but write first.

text = 'Sample Text to Save\nNew line!'

# notifies Python that you are opening this file, with the intention to write

saveFile = open('exampleFile.txt','w')

# actually writes the information

saveFile.write(text)

# It is important to remember to actually close the file, otherwise it will

# hang for a while and could cause problems in your script

saveFile.close()

Saving data to a file is as simple as that! Just don't forget to close it or you will likely find yourself having

lots of i/o issues in your programs!

Appending Files Now we get to appending a file in python. Note that writing will clear the file and write to it just the data

you specify in the write operation. Appending will simply take what was already there, and add the new

data to it.

That said, when you actually go to add to the file, you will still use ".write." You only specify that you will

be appending instead of writing when you open the file and specify your intentions.

Let's look at an example:

# so here, generally it can be a good idea to start with a newline, since

# otherwise it will append data on the same line as the file left off.

# you might want that, but I'll use a new line.

# another option used is to first append just a simple newline

# then append what you want.

appendMe = '\nNew bit of information'

appendFile = open('exampleFile.txt','a')

appendFile.write(appendMe)

appendFile.close()

What will happen here is, if exampleFile.txt already exists, the appendMe line will be added to it.

If that file does not already exist, then it will be created.

Reading from Files Now that we know how to write and append to files, we might want to learn how to read data from files

into the Python program. Doing this is quite simple, and has very similar syntax.

# similar syntax as you've seen, 'r' for read. You can just throw a .read()

at

# the end, and you get:

readMe = open('exampleFile.txt','r').read()

print(readMe)

Often times, people are reading something with many lines into memory. Maybe it's a list of names, or

something like that. We can then use ".readlines()" to help us split all of this up into a Python list for us.

# this will instead read the file into a python list.

readMe = open('exampleFile.txt','r').readlines()

print(readMe)

Classes You can think of classes as groupings of functions, usually. Classes quickly work their way into

"intermediate" programming, so hopefully I can just help you understand how they work and how to

follow code that uses them.

Classes are the backbone to Object Oriented Programming, or OOP. As you get comfortable with

Python, classes can become an absolutely integral part of our programs.

class calculator:

def addition(x,y):

added = x + y

print(added)

def subtraction(x,y):

sub = x - y

print(sub)

def multiplication(x,y):

mult = x * y

print(mult)

def division(x,y):

div = x / y

print(div)

Here are some examples using the above code in the interpreter:

>>> calculator.subtraction(5,8)

-3

>>> calculator.multiplication(3,5)

15

>>> calculator.division(5,3)

1.6666666666666667

>>> calculator.addition(5,2)

7

>>>

Classes are used for Object Oriented Programming, or OOP.

The statistics module comes with an assortment of goodies: Mean, median, mode, standard deviation,

and variance.

These are all fairly straight forward to use, here and some simple examples:

import statistics

example_list = [5,2,5,6,1,2,6,7,2,6,3,5,5]

x = statistics.mean(example_list)

print(x)

y = statistics.median(example_list)

print(y)

z = statistics.mode(example_list)

print(z)

a = statistics.stdev(example_list)

print(a)

b = statistics.variance(example_list)

print(b)

As you can see, you just simply pass a list through the module's function, and you're output is the

answer. Here, we're saving the output to a variable, and then we're just printing out the variable. In

normal circumstances, you'd probably continue doing things with it.

Here, we've seen how simple importing and using modules can be, but there are a lot of other options

when it comes to how we import things.

Module import Syntax Now that we've used a module, statistics, it would be a good time to explain some import syntax

practices. As with many things in programming, there are many ways to import modules, but there are

certainly some best practices.

So first, when you import a module, you are basically loading that module into memory. Think of a

module like a script. Many if not most modules are just a single python script. So, when you go to import

it, you use the file name. This can help keep code clean and easy to read. Many python developers just

program everything in 1 script. Other developers, say from a language like java are going to be very used

to doing lots of imports with a file for each type of job that's happening. Just like there are many ways to

import, there are many more ways to program.

So let's talk about basic importing:

import statistics

Above, we have referenced the statistics module and loaded it into memory under the statistics object.

This will allow us to reference any of the functions within the statistics module. To do so, we will need to

mention statistics, followed by a period, then the function name. A simple exhibition of the mean

function from statistics could look like this:

import statistics

example_list = [5,2,5,6,1,2,6,7,2,6,3,5,5]

print(statistics.mean(example_list))

That is the simplest way to import and use modules, but there are many other methods. Sometimes,

however, you will see people use the "as" statement in their imports. This will allow you to basically

rename the module to whatever you want. People generally do this to shorten the name of the module.

Matplotlib.pyplot is often imported as plt and numpy is often imported as np, for example.

import statistics as s

print(s.mean(example_list))

Above, we've imported statistics as the letter 's.' This means whenever we wish to reference the statistics module, we just need to type 's' instead of statistics.

What if you don't even want to type that S though? Well there's an app for that!

You can just import each function within the module you plan to use:

from statistics import mean

# so here, we've imported the mean function only.

print(mean(example_list))

# and again we can do as

from statistics import mean as m

print(m(example_list))

Above, you can see that we no longer had to type any reference to the statistics module, then you saw

that we could even import the functions "as" something else.

What about more functions?

from statistics import mean, median

# here we imported 2 functions.

print(median(example_list))

What if we want to use the as as well?

from statistics import mean as m, median as d

print(m(example_list))

print(d(example_list))

What if we want to just import everything from statistics like we did initially, but we don't want to type the statistics because we have fat fingers and this will just slow us down?

from statistics import *

print(mean(example_list))

Lists and Tuples

Our focus here is the difference between Python lists and tuples. Often confused, due to their

similarities, these two structures are substantially different.

A tuple is an assortment of data, separated by commas, which makes it similar to the Python list, but a

tuple is fundamentally different in that a tuple is "immutable." This means that it cannot be changed,

modified, or manipulated. A tuple is typically used specifically because of this property. A popular use

for this is sequence unpacking, where we want to store returned data to some specified variables.

Something like:

def example():

return 15, 12

x, y = example()

print(x,y)

# in the above case, we have used a tuple and cannot modify it... and

# we definitely do not want to!

If you notice, the tuple had no brackets around it at all. If there are no encasing brackets or braces of

any type, then Python will recognize the data as a tuple. Tuples also can have curved brackets like "(" or

")"

Next, we have the far more popular Python list. To define a list, we use square brackets. A Python list

acts very much like an array in other languages like php.

Here's an example of a list and an example use:

x = [1,3,5,6,2,1,6]

'''

You can then reference the whole list like:

'''

print(x)

# or a single element by giving its index value.

# index values start at 0 and go up by 1 each time

print(x[0],x[1])

List Manipulation In this section we see list manipulation. This includes adding things to the end, inserting them into

specific positions, removing things, finding data, counting the number of occurrences, sorting, and

reversing the data.

All of the above are very common operations with lists, and all of them are built into Python 3 for ease

of use.

Keep in mind that lists are mutable, and using these functions change the list.

Here is some example code of list manipulation:

Since lists are mutable, this means that we will be using lists for things where we might intend to

manipulate the list of data, so how can we do that? Turns out we can do all sorts of things.

We can add, remove, count, sort, search, and do quite a few other things to python lists.

# first we need an example list:

x = [1,6,3,2,6,1,2,6,7]

# lets add something.

# we can do .append, which will add something to the end of the list, like:

x.append(55)

print(x)

Above, we took a list, and added to the end of the list with .append. Remember append with files? Same

thing, .append() will add to the end of a list.

What if you have an exact place that you'd like to put something in a list, instead of just at the very end?

x.insert(2,33)

print(x)

Here we say that we want to insert, at the index of 2, the number 33. The reason that went in the 3rd

place, again, is because we start at the zero element, then go 1, 2...etc with lists.

Now we can remove things. .remove() will remove the first instance of the value in the list. If it doesn't

exist, there will be an error:

x.remove(6)

print(x)

Next, remember how we can reference an item by index in a list? like:

print(x[5])

Well, we can also search for this index, like so:

print(x.index(1))

Now here, we can see that it actually returned a 0, meaning the first element was a 1... when we knew

there was another with an index of 5. So, instead we might want to know before-hand how many

examples there are.

print(x.count(1))

We see there are actually 2 of them. We can also sort the list:

y = ['Jan','Dan','Bob','Alice','Jon','Jack']

y.sort()

print(y)

y.reverse()

print(y)

Multi-dimensional lists

Multi-dimensional lists are lists within lists, or lists within lists within lists... you get the point. It can get

very confusing very fast, but it is good to know that it is an option. Usually a dictionary will be the better

choice rather than a multi-dimensional list in Python, but, if you are familiar with multi-dimensional

arrays in other languages, you might want to continue that concept in Python.

Lists that we have covered so far have all been 1 dimensional, but you can have lists within lists within

lists within lists if you want.

We already know how to reference elements in a list, we can do:

x = [[2,6],[6,2],[8,2],[5,12]]

print(x[2])

You will get [8,2] as output, remember the index usually starts with ‘0’.

We can also take this deeper since we have more dimensions now:

print(x[2][1])

This can go on indefinitely with very thick lists. You might see how this can quickly get messy, so let's

consider how to properly display lists in code that have many dimensions. You might not typically hard

code multi-dimensional lists, but there are some instances where you will.

y = [[5,2],

[6,2],

[3,1],

[12,6]

]

This is slightly cleaner, and python automatically understands it:

print(y)

Reading CSV File in Python 3: We saw that csv is one file type to save data. How to read CSV data in from a file and then use it in

Python?

For this, we use the csv module. CSV literally stands for comma separated variable, where the comma is

what is known as a "delimiter." While you can also just simply use Python's split() function, to separate

lines and data within each line, the CSV module can also be used to make things easy.

Here is the sample code that matches the video:

Example CSV file data: (Copy and paste it in text file and save it with csv extension)

1/2/2014,5,8,red

1/3/2014,5,2,green

1/4/2014,9,1,blue

Next, let's cover the reading of CSV files into memory:

import csv

with open('example.csv') as csvfile:

readCSV = csv.reader(csvfile, delimiter=',')

for row in readCSV:

print(row)

print(row[0])

print(row[0],row[1],row[2],)

Above, we've shown how to open a CSV file and read each row, as well as reference specific data on

each row.

Next, we will show how to pull out specific data from the spreadsheet and save it to a list variable:

import csv



dates = []

colors = []

for row in readCSV:

color = row[3]

date = row[0]

dates.append(date)

colors.append(color)

print(dates)

print(colors)

Once we have this data, what can we do with it? Maybe we are curious about what color something was

on a specific date.

import csv



dates = []

colors = []

for row in readCSV:

color = row[3]

date = row[0]

dates.append(date)


print(dates)

print(colors)

# now, remember our lists?

whatColor = input('What color do you wish to know the date of?:')

coldex = colors.index(whatColor)

theDate = dates[coldex]

print('The date of',whatColor,'is:',theDate)

Try and Except Error handling In this section we cover the Try and Except statements, which are used for error handling. These

statements work similarly to the if-else, where if the Try runs, the except will not run. If the Try fails,

then the exception will run with the error that was just generated in the try. Try and Except is mainly

used to handle failures in code, which result in errors. With handling exceptions, you can keep your code

running when it would otherwise grind to a catastrophic halt from an error. You can also use error

handling to log problems in your code, or to even attempt to remedy the problem as a part of the

program.

Use the previous CSV data

Now we need to learn how to handle errors in Python. As you can see, if you do not have any error

handling, your program or script will just stop completely. This is not likely to be desired! First, we want

to figure out how to handle errors, which is really just treating the symptom to the problem, not really

solving the problem. Then, we want to learn how to avoid these sorts of problems to begin with, using

proper logic in our programs.

import csv



dates = []

colors = []

for row in readCSV:

color = row[3]

date = row[0]

dates.append(date)


print(dates)

print(colors)

We can put the "try" statement in quite a few places to solve our problem. I like to put the try statement

in front of any new "block" of logic. Here, our logic block begins by asking the user to enter some input.

This should immediately be a cause for concern, since we're opening a "door" to our program, and we

might receive some unwanted input.

A great example of this is input fields on websites. Say you have a log-in with username and password

field. Generally, you expect the user to enter a username and a password. First off, they might enter a

non-existent username and password. Either they do not have one, they forgot it, or accidentally made a

mistake in the input.

Sometimes, input isn't as innocent as this, as people sometimes have more sinister intentions. Using

what is known as SQL injection, hackers can attempt to either gain access to your database or

administration, or even wreak havoc on your website by modifying or just plain dropping tables in your

database. Instead of entering usernames and passwords, hackers can enter SQL queries. If not handled

right, your back-end code might actually go to run the SQL statement the hacker intends. The basic SQL

injection usually starts with closing off the SQL statement you intended to run from your field, then

begins a brand new query from the field. It looks nothing like a username or a password to a person, but

your system may still blindly run it. This is a prime example where not only error handling is necessary to

know, but also proper logic is imperative.

try:





# in python 2, this is read exception Exception, e. It's just helpful

# to know this for porting old scripts if you need to.

except Exception as e:

print(e)

''' So this will try a block of code, and, if there is an exception, it

will continue to run...

'''

print('Stillllllll running though!')

This shown, we should always treat try and exception handling as a last resort, or a final point of failure.

We should really instead code in a conditional before the exception is thrown, something like:



dates = []

colors = []

for row in readCSV:

color = row[3]

date = row[0]

dates.append(date)


print(dates)

print(colors)

# we could put the try anywhere. The weak point, however, starts

# in my opinion immediately when we accept user input... no longer

# is this is a closed-program, so I would personally code this block

# here, but you could put the try right about the print statement

# of where we search for the color and we KNOW it will throw an error

# if not in the list.

try:


if whatColor in colors:




else:

# now we can handle a specific scenario, instead

# of handling it with a "catch-all" error.

# now we have error handling and

# proper logic. Yay.

print('This color was not found.')

# in python 2, this is read exception Exception, e. It's just helpful

# to know this for porting old scripts if you need to.

except Exception as e:

print(e)

Above, we've not only handled an error if we get one, but we also have coded in proper logic. Just as a

quick note, the above logic will absolutely not stop an SQL injection. I am just using that as an example

of why logic is necessary!

Dictionaries Dictionaries are a data structure in Python that are very similar to associative arrays. They are non-

ordered and contain "keys" and "values." Each key is unique and the values can be just about anything,

but usually they are string, int, or float, or a list of these things.

Dictionaries are defined with {} curly braces.

Here is the sample code:

'''

One of the most useful data types in python is the python

dictionary.

If you are familiar with other languages, think of it like an associative

array.

The idea of the dictionary is to have what are called keys and values.

Despite

being ordered if you print a dictionary out, there is no actual

order to dictionaries.

All keys are unique

So before we used two lists and assumed their association, searched for

index,

and found information about 1 item in 1 list from another.

Now here, everything is contained in the same location, and makes more sense

Let us show an example:

'''

# Dictionary of names and ages.

exDict = {'Jack':15,'Bob':22,'Alice':12,'Kevin':17}

print(exDict)

How old is Jack?

print(exDict['Jack'])

We find a new person that we want to insert:

exDict['Tim'] = 14

print(exDict)

Tim just had a birthday though!

exDict['Tim'] = 15

print(exDict)

Then Tim died.

del exDict['Tim']

print(exDict)

Next we want to track hair color:

exDict = {'Jack':[15,'blonde'],'Bob':[22,

'brown'],'Alice':[12,'black'],'Kevin':[17,'red']}

print(exDict['Jack'][1])

Built in functions In this section, we cover a handful of the built-in functions with Python 3. For a full list, see:

https://docs.python.org/3/library/functions.html

We cover absolute value (abs()), the help() functions, max(), min() ...which are how to find maximum

and minimum of a list, how to round a number with round(), as well as ceil() and floor(), even though

these last two are NOT built in, it just seemed like a good time to bring them up. Finally, we cover

converting floats, ints, and strings to and from each other.

Absolute Values:

exNum1 = -5

exNum2 = 5

print(abs(exNum1))

if abs(exNum1) == exNum2:

print('True!')

The Help function:

This is probably one of the most under-utilized commands in Python, many people do not even know

that it exists. With help(), you can type it with empty parameters to engage in a search, or you can put a

specific function in question in the parameter.

help()

or

import time

# will work in a typical installation of Python, but not in the embedded

editor

help(time)

Max and Min:

How to find the maximum or highest number in a list… or how to find the lowest or minimum number in

a list.

exList = [5,2,1,6,7]

largest = max(exList)

print(largest)

smallest = min(exList)

print(smallest)

Rounding:

Rounding will round to the nearest whole. There are also ways to round up or round down.

x = 5.622

x = round(x)

print(x)

y = 5.256

y = round(y)

print(y)

Converting data types:

Many times, like reading data in from a file, you might find the datatype is incorrect, like when we mean

to have integers, but they are currently in string form, or vice versa.

Converting a string to an integer:

intMe = '55'

intMe = int(intMe)

print(intMe)

Converting and integer to a string:

stringMe = 55

stringMe = str(stringMe)

print(stringMe)

Converting an integer to a float:

floatMe = 55

floatMe = float(floatMe)

print(floatMe)

You can also convert floats to strings, strings to floats, and more. Just make sure you do a valid

operation. You still cannot convert the letter h to a float.

OS Module The main purpose of the OS module is to interact with your operating system. The primary use I find for

it is to create folders, remove folders, move folders, and sometimes change the working directory. You

can also access the names of files within a file path by doing listdir().The os module is a part of the

standard library, or stdlib, within Python 3. This means that it comes with your Python installation, but

you still must import it.

Sample code using os:

import os

All of the following code assumes you have os imported. Because it is not a built-in function, you must

always import it. It is a part of the standard library, however, so you will not need to download or install

it separately from your Python installation.

curDir = os.getcwd()

print(curDir)

The above code will get your current working directory, hence "cwd."

To make a new directory:

os.mkdir('newDir')

To change the name of, or rename, a directory:

os.rename('newDir','newDir2')

To remove a directory:

os.rmdir('newDir2')

With the os module, there are of course many more things we can do. In many scenarios, however, the

os module is actually becoming outdated, as there is a superior module to get the job done. It is still a

good idea to at least know some of the basics of the os module. I especially like to use it to create

directories. If you ever create a setup.py file, the creation of directories and the placing of files within

them will be essential.

Matplotlib

One of the most popular uses for Python is data analysis. Naturally, data scientists want a way to

visualize their data. Either they want to see it for themselves to get a better grasp of the data, or they

want to display the data to convey their results to someone. With Matplotlib, arguably the most popular

graphing and data visualization module for Python, this is very simplistic to do.

Once you have Matplotlib installed, be sure to open up a terminal or a script, type:

import matplotlib

Make sure there are no errors on the import. If there are, read the error. Most often, either the bit

version does not match (64 bit vs 32 bit), or you are missing a package like dateutil or pyparsing.

Once you can successfully import matplotlib, then you are ready to continue.

Here's some basic code to generating one of the most simple graphs that we can, it will take us only 3

lines.

#Importing pyplot

from matplotlib import pyplot as plt

#Plotting to our canvas

plt.plot([1,2,3],[4,5,1])

#Showing what we plotted

plt.show()

As you progress with Matplotlib, it might be useful to understand how it works fundamentally. This

process is true with a lot of computer graphics processes. First, you have some data, then you "draw"

that data to a canvas of some sort, but it is only in the computer's memory. Once you've drawn that

data, you can then "show" that data. This is so the computer can first draw everything, and then

perform the more laborious task of showing it on the screen.

So, with the code above, we just import pyplot from matplotlib, we use pyplot to "plot" some data to

the canvas in memory, then we use plt, which is pyplot, to show what we've got.

Now, of course, there are some problems with our graph. First off, we learned in school that we're

supposed to put labels on each axis and that we need a title to our graph or chart. Next, in terms of

programming, it is unlikely that you will actually be filling in data to the plt.plot() function. Instead, you

will, or at least you should, be only passing variables into it. Like plt.plot(x,y). So now let us show plotting

variables as well as adding some descriptive labels and a good title!


x = [5,8,10]

y = [12,16,6]

plt.plot(x,y)

plt.title('Epic Info')

plt.ylabel('Y axis')

plt.xlabel('X axis')

plt.show()

Next up, however, our graph is, well, ugly. If you want to learn all of the ins and outs to heavily

customizing your graphs, then you will definitely want to check out the Matplotlib help documents

about the functions and try to do it yourself.

For example, if you used style package:


from matplotlib import style

style.use('ggplot')

x = [5,8,10]

y = [12,16,6]

x2 = [6,9,11]

y2 = [6,15,7]

# can plot specifically, after just showing the defaults:

plt.plot(x,y,linewidth=5)

plt.plot(x2,y2,linewidth=5)




plt.show()

Here, as you can see, the only reference to styling that we've made is the style.use() function, as well as

the line width changes.

Let's add a legend to our chart, and, since it is simple enough, let's learn about grid lines too:



style.use('ggplot')

x = [5,8,10]

y = [12,16,6]

x2 = [6,9,11]

y2 = [6,15,7]

Up to this, everything is about the same, but now you can see we've added another parameter to our

plt.plot(), which is "label." Just to clarify, for those who are not yet totally comfortable with the notion

of default parameters in functions, some people may be curious about why we are able to plot the x, y,

and color variable without any sort of assignment, but then we have to assign label and linewidth. The

main reason here is because there are many parameters to pyplot.plot(). It is really easy to forget their

order. X, y, and color is fairly easy to remember the order, people are good at remembering orders of

three. After that, the chances of forgetting the proper order get quite high, so it just makes sense. There

are also many parameters to edit, so we just call them specifically. Anyway, we can see here that we

added a "label," so matplotlib knows what to call the line. This doesn't quite yet give us a legend,

however. We need to call plt.legend(). It's important to call legend AFTER you've plotted what you want

to be included in the legend.

plt.plot(x,y,'g',label='line one', linewidth=5)

plt.plot(x2,y2,'c',label='line two',linewidth=5)




plt.legend()

plt.grid(True,color='k')

plt.show()

Bar charts and scatter plots

Bar charts with matplotlib are basically 1 slight change, same with scatter plots. The only major change I

like to make to bar charts is to center them, and that's about it:



style.use('ggplot')

x = [5,8,10]

y = [12,16,6]

x2 = [6,9,11]

y2 = [6,15,7]

plt.bar(x, y, align='center')

plt.bar(x2, y2, color='g', align='center')




plt.show()

So, here, we can see instead of plt.plot(), we've used plt.bar(). We also used a new parameter called

align, and made it align centered. How about scatter plots? Super easy, we'll just change .bar() to

.scatter(), and remove our align parameter:



style.use('ggplot')

x = [5,8,10]

y = [12,16,6]

x2 = [6,9,11]

y2 = [6,15,7]

plt.scatter(x, y)#, align='center')

plt.scatter(x2, y2, color='g')#, align='center')




plt.show()

Now try these functions with our Prime database.

Using Numpy NumPy: array processing for numbers, strings, records, and objects.

NumPy is a general-purpose array-processing package designed to efficiently manipulate large multi-

dimensional arrays of arbitrary records without sacrificing too much speed for small multi-dimensional

arrays. NumPy is built on the Numeric code base and adds features introduced by numarray as well as

an extended C-API and the ability to create arrays of arbitrary type which also makes NumPy suitable for

interfacing with general-purpose data-base applications.

1,5

2,7

3,8

4,3

5,5

6,6

7,3

8,7

9,2

10,12

11,5

12,7

13,2

14,6

15,9

16,2

Save the above data as "exampleFile.csv" in the root directory as your current script. Then run this code:



import numpy as np

style.use('ggplot')

x,y = np.loadtxt('exampleFile.csv',

unpack=True,

delimiter = ',')

plt.plot(x,y)




plt.show()

Here, our major new things are importing numpy, and then using numpy's loadtxt function. Loadtxt can

be used to load more than just .txt files. It's just load things with text, that's all. Here, we are unpacking

the contents of exampleFile.csv, using the delimiter of a comma. It's important to note here that you

MUST unpack the exact same number of columns that will come from the delimiter that you state. If

not, you'll get an error.

Pie Chart Pie Chart is used to show parts to the whole, and often a % share. Luckily for us, Matplotlib handles the

sizes of the slices and everything, we just feed it the numbers.

import matplotlib.pyplot as plt

slices = [7,2,2,13]

activities = ['sleeping','eating','working','playing']

cols = ['c','m','r','b']

plt.pie(slices,

labels=activities,

colors=cols,

startangle=90,

shadow= True,

explode=(0,0.1,0,0),

autopct='%1.1f%%')

plt.title('Interesting Graph\nCheck it out')

plt.show()

Within the plt.pie, we specify the "slices," which are the relevant sizes for each part. Then, we specify

the color list for the corresponding slices. Next, we can optionally specify the "Start angle" for the graph.

This lets you start the line where you want. In our case, we chose a 90 degree angle for the pie chart,

which means the first division will be a verticle line. Next, we can optionally add a shadow to the plot for

a bit of character, and then we can even use "explode" to pull out a slice a bit.

We have four total slices, so, with explode, if we didn't want to pull out any slices at all, we would do

0,0,0,0. If we wanted to pull out the first slice a bit, we would do 0.1,0,0,0.

Finally, we do autopct to optionally overlay the percentages on to the graph itself.

Graph from File: Many times, people want to graph data from a file. There are many types of files, and many ways you

may extract data from a file to graph it. Here, we'll show a couple of ways one might do this. First, we'll

use the built-in csv module to load CSV files, then we'll show how to utilize NumPy, which is a third-

party module, to load files.


import csv

x = []

y = []

with open('example.txt','r') as csvfile:

plots = csv.reader(csvfile, delimiter=',')

for row in plots:

x.append(int(row[0]))

y.append(int(row[1]))

plt.plot(x,y, label='Loaded from file!')

plt.xlabel('x')

plt.ylabel('y')


plt.legend()

plt.show()

Here, we open a sample file, which contains the following data:

1,5

2,3

3,4

4,7

5,4

6,3

7,5

8,7

9,4

10,4

Next, we use the csv module to read in the data. The csv reader automatically splits the file by line, and

then the data in the file by the delimiter we choose. In our case, this is a comma. Note: the "csv" module

and the csv reader does not require the file to be literally a .csv file. It can be any text file that simply has

delimited data.

Once we've done this, we store the elements with an index of 0 to the x list and the elements with an

index of 1 to the y list. After this, we're all set and ready to plot, then show the data.

While using the CSV module is completely fine, using the NumPy module to load our files and data is

likely to make more sense for us down the line. Once you have NumPy, you can write code like:


import numpy as np

x, y = np.loadtxt('example.txt', delimiter=',', unpack=True)

plt.plot(x,y, label='Loaded from file!')

plt.xlabel('x')

plt.ylabel('y')


plt.legend()

plt.show()

How to use Internet Data for Graphing Aside from loading data from the files, another popular source for data is the internet. We can load data

from the internet from a variety of ways, but, for us, we're going to just simply read the source code of

the website, then use simple splitting to separate the data.


import numpy as np

import urllib

import matplotlib.dates as mdates

def graph_data(stock):

stock_price_url =

'http://chartapi.finance.yahoo.com/instrument/1.0/'+stock+'/chartdata;type=qu

ote;range=10y/csv'

source_code = urllib.request.urlopen(stock_price_url).read().decode()

stock_data = []

split_source = source_code.split('\n')

for line in split_source:

split_line = line.split(',')

if len(split_line) == 6:

if 'values' not in line:

stock_data.append(line)

There's a lot going on here. First, we see the imports. Pyplot is imported as usual, then numpy, then

urllib for accessing the internet, and then matplotlib.dates as mdates, which is useful for converting date

stamps to dates that matplotlib can understand.

Next, we begin to build our "graph_data" function. In here, we first define the url that contains the stock

data. After that, we write some urllib code to access that url, then read, with .read, the source code,

then we go ahead and decode the data.

Then, we define an empty list, which is where we'll be placing the stock data shortly, and we also begin

to split up the data with the split_source variable, splitting by new lines.

Now, if you go to look at the source code, replacing the "+stock+" in the url with a stock, like AAPL, you

can see that most of the page data is indeed stock pricing information, but there is some header

information there that we need to filter out. To do this, we'll just use some rudimentary filtration,

checking to make sure there are 6 data points per line, and then making sure the term "values" isn't in

the line.

Now, we have the data parsed out, and we're ready to munch on it. We're going to use NumPy for this:

date, closep, highp, lowp, openp, volume = np.loadtxt(stock_data,

delimiter=',',

unpack=True,

# %Y = full year.

2015

# %y = partial year

15

# %m = number month

# %d = number day

# %H = hours

# %M = minutes

# %S = seconds

# 12-06-2014

# %m-%d-%Y

converters={0:

bytespdate2num('%Y%m%d')})

What we are doing here, is unpacking these six elements to six variables, with numpy's loadtxt function.

The first parameter here is stock_data, which is the data we're loading. Then, we specify the delimiter,

which is a comma in this case, then we specify that we indeed want to unpack the variables here not just

to one variable, but to this group of variables we've defined. Finally, we use the optional "converters"

parameter to specify what element we want to convert (0), and then how we want to do that. We pass a

function called bytespdate2num, which doesn't quite exist yet, but we'll write that next.

def bytespdate2num(fmt, encoding='utf-8'):

strconverter = mdates.strpdate2num(fmt)

def bytesconverter(b):

s = b.decode(encoding)

return strconverter(s)

return bytesconverter

This function takes the data, decodes the data based on the encoding, then it returns that.

Here is the complete program:


import numpy as np

import urllib

import matplotlib.dates as mdates

def bytespdate2num(fmt, encoding='utf-8'):

strconverter = mdates.strpdate2num(fmt)

def bytesconverter(b):

s = b.decode(encoding)

return strconverter(s)

return bytesconverter

def graph_data(stock):

stock_price_url =

'http://chartapi.finance.yahoo.com/instrument/1.0/'+stock+'/chartdata;type=qu

ote;range=10y/csv'

source_code = urllib.request.urlopen(stock_price_url).read().decode()

stock_data = []

split_source = source_code.split('\n')

for line in split_source:

split_line = line.split(',')

if len(split_line) == 6:

if 'values' not in line and 'labels' not in line:

stock_data.append(line)

date, closep, highp, lowp, openp, volume = np.loadtxt(stock_data,

delimiter=',',

unpack=True,

# %Y = full year.

2015

# %y = partial year

15

# %m = number month

# %d = number day

# %H = hours

# %M = minutes

# %S = seconds

# 12-06-2014

# %m-%d-%Y

converters={0:

bytespdate2num('%Y%m%d')})

plt.plot_date(date, closep,'-', label='Price')

plt.xlabel('Date')

plt.ylabel('Price')


plt.legend()

plt.show()

graph_data('TSLA')

You can do lots of customization for the above graph. But I leave it to self learning.

Pandas: Data manipulation, visualization, and analysis with for Python

We’ll see this topic in the next class.

python 3 programming

Documents