Decision Time & Machine Learning

 Life, Python  Comments Off on Decision Time & Machine Learning
Oct 262020
 

It’s decision time. I have spent the past 6 months or so learning the Python. Other than some messing around on my Commodore 64 when I was a teen, I started this journey with zero knowledge about coding. A variety of background factors had driving me to the point in my life where significant changes were required and, after some initial doubt as to whether or not it was a feasible choice for me, I landed on coding as a means of bringing about the required change.

The landscape has changed beyond all recognition since the last time I undertook serious study back in the nineties. The amount of information now available to the potential student can be overwhelming and demands a higher degree of focus than I had previously had to muster. So many coding blogs, with so many links to other coding blogs. So many YouTube videos with an ever-distracting plethora of comments to get lost in and related videos suggested. So many online courses on so many online learning platforms.

Even when trying to decide on something as simple as a book, where you might think you can escape the noise, you face the same issue of being presented with recommended books that others bought book-bundles that might appeal as well as upsells at the checkout.

The Paradox of Choice

Suffice it to say, it takes an iron will to navigate this landscape without pouring hours of precious time down the drain. It’s something that I am still working on and adjusting to in my efforts to reach my Python-related goals and regularly reminds me of the paradox of choice in modern society.

So, to even reach the first step of deciding on what language to learn was small struggle. Being frank, the decision was based in part on the fact that Python is considered one of the easier languages to learn and a good place to start coding as a springboard to other languages. Beyond that, I wanted to learn a language that can be used in a variety of applications and it turns out that, as a general purpose language, Python can be used in all sorts of ways. Game development, web development and apps, web-scraping, pen-testing, data analysis, machine learning / AI etc. etc. It’s used by tech-giants like Google, Facebook, Dropbox and a long list of others. There’s also an active, friendly and supportive community there to help foster the talent.

Now, I am sure that most of the same can be said of other languages, but ultimately, Python was were I settled. Having reached that point, (just like all other Python neophytes) I was again faced with the paradox of choice and an overwhelming amount of options. Where to start? Which direction to take? Which YouTube channel(s) to subscribe to? Should I bother to sign up to email lists and which ones are worth the time? Where to spend hard-earned cash on books and/or courses. Nowadays, of course, even selecting a book requires a second choice of whether to invest in the digital or physical format.

Initial Resources

After much deliberation and improving my ability to reduce noise and focus on what matters, I bought the kindle version of Eric Mattes’ fine text book Python Crash Course and diligently worked my way through it, absorbing and retaining as much as I could. On completing the book, two things struck me:

  1. It’s an excellent introductory resource and gave me a solid base to build on.
  2. The Kindle format may be great for the latest page-turner, but IMHO it’s a poor substitute for the physical text book.

Another thing that became clear to me was that, in the world of computer programming, it’s all about using code to solve problems. This lead me to a point where I was very keen on developing problem solving skills. Codewars offered me a free and goal orientated means of getting better at writing code with the specific intention of solving a problem. Here’s my current ranking:

Codewars Ranking

Given that my ultimate goal in learning to code is to gain employment using my Python skillset, it became clear that a high ranking on Codewars would not be enough. The internet reliably informed me that I needed to be able to demonstrate my skills with a coding project. That way, potential employers would be able to deduce whether I would be worth employing or not.

Having spent time online (blogs, YouTube) looking at potential projects, I came to the conclusion that I needed to upskill if I were to be able to code up something unique and not just copy and paste an existing project. After short deliberation, I decided on Ardit Sulce’s course on Udemy, as the focus was on putting together a variety of real world applications. Turned out that it was an excellent resource and delivered exactly what I needed – information that I could use in putting together my own project.

Decision Time

In my initial post on this blog, I provided details on my first project Fan Fun – a ‘fun’ quiz program that combines several Python libraries – web scraping with BeautifulSoup & Requests, GUI with TKinter, text to speech with pyttsx. I had a lot of fun putting it together and learned that, no matter where you are in the sphere of programming excellence, just start. Make a start, put a project together and learn as you go. Otherwise, you’ll never make the jump.

Which brings me to the point of this whole post – decision time. Essentially, I reached a point where I needed to decide where specifically to focus my attention with Python. When I completed Fan Fun, there was a sense of “OK, what now?”. I felt that I could put together another project as proof of competency, but whatever I did next needed greater intention behind it. I needed to decide where I wanted to go with Python.

From the outset, Machine Learning and AI had appealed to me. The more I looked into it and the requirements needed to become a successful machine learning engineer, the more it appealed. I had looked into other options, but there was no spark there. Nothing that left me feeling as excited as the concept of AI. Machine Learning is something that’s been with us for a few decades already, but with the onset of greater computing power, lower software/hardware costs, IoT, Big Data etc, it’s becoming more and more a part of life and will most certainly continue to be so. And that’s something I want to be part of.

And that’s where my journey so has brought me. I have decided to focus on developing the skills needed in data analysis and machine learning. It’s not going to be easy, but then nothing worthwhile ever is. My journey started last month when I invested in Daniel Bourke’s excellent course on Udemy.

I have also invested in physical text books that will help develop a robust understanding of data analysis and machine learning:

  • Python For Data Analysis by Wes Mckinney
  • Grokking Deep Learning by Andrew Trask
  • Competing in the Age of AI by Marco Iansiti & Karim Lakhani

Machine Learning and Future Posts

Going forward, I expect to be posting about projects relating to processing data with MatPlotLib, Pandas and Numpy, getting comforatable with SciKitLearn’s ‘out-of-the-box’ estimators and, in the not too distant future, things like deep learning, neural networks and TensorFlow. I also expect to be paying close attention to Daniel Bourke’s blog and KDNuggets.com to ensure the continued consumption of quality content on machine learning and AI.

Something tells me that this is just the beginning…

Fan Fun – First Python Project

 Projects, Python  Comments Off on Fan Fun – First Python Project
Sep 102020
 

I am reliably informed by the internet that developing and coding a python project is an ideal way to simultaneously escape tutorial hell and augment one’s coding skillset. So, armed with a couple of months’ study of Python basics, I decided to commit to completing my first project and to publish the results on my blog. Hence this blogpost.

Solitary First Steps & Goals

Going into this, I had two goals: I wanted to create something fun and I wanted to develop my coding, web-scraping and GUI skills. With that in mind, I brushed up on scraping basics and how to work with BeautifulSoup to extract desired data from target URLs. It was then that I chanced upon this potential project and decided to take up the challenge to “take things to the next level”. Initial thoughts were that I would add a GUI, include user input to specify a band/artist and perhaps include text to voice to enunciate the program’s output.

Fine Tuning the Concept

On further reflection and, to make the project more unique and a bit more fun, I decided to integrate a few more things and ended up with the following concept;

  1. User enters name of favorite band or artist into input field of GUI
  2. Python takes the input, formulates URL
  3. Paired with BeautifulSoup, Python then sends a get request, using the URL defined and created in step two
  4. The target URL is then scraped for and Python selects a track from an album
  5. Python then collects random lyrics from a randomly selected album track
  6. Using pytnndy, the selected lyrics are enunciated
  7. The user guesses the name of the track
  8. Python announces whether the guess was correct or incorrect

Let’s take a look at the code that I wrote to achieve these steps and create a music-based quiz program, starting with the libraries imported:

python libraries

Tkinter is the library I elected to import to allow for GUI creation, BeautifulSoup is my go-to library for web-scraping. Requests facilitated get requests to the target site and urllib automated the opening and reading of specific webpages on the target site. Random allowed me to write code that could randomly select a track from albums by the selected artist/band and also to randomly select lyrics from said track. It was also used to select a compliment/insult at random from a list, depending on whether the user correctly guesses the track or not. Finally, pyttsx3 was the library I chose to execute the text to voice function that I thought would add a cool twist to the program. It was used to enunciate the track lyrics, as well as to announce whether the user guessed correctly or not.

GUI Sections

I elected to work on the code for the GUI first and then to add functions triggered by input and buttons assigned with commands. After sorting out the root (root = tk.Tk()), the first step was to define the canvas size and background image:

The next step was to position the required TKinter widgets within the frame. I chose to work with the .place() function to accomplish this, as I find it offers more flexibility over the alternative .pack() and .grid() functions. I decided to split the GUI into five sections, as follows:

Section 1 – How user defines difficulty level of quiz

Section 2 – Where user enters favorite band/artist name

Section 3 – Where user enters guess

Section 4 – Where quiz result is displayed

Section 5 – Quit Button

The option of offering the user three difficulty levels was not part of the original concept. It was an after-thought that I liked and that decided to keep. Section 1 includes one label with instructive text and three buttons. Easy, Medium & Hard. Each of the three buttons is associated with a function and passes parameter data to its function using a command / lambda combination:

To facilitate Sections 2 & 3 of the GUI, I needed three widgets per section – an instructive label, an input field and a button to trigger the associated function. Here’s the code I put together for those:

For both the artist input and guess input sections, the input is taken from the entry field and passed to an associated function using command. In order to pass that data to the respective function arguments, it was necessary to create a simple lambda function, as the command keyword cannot be used to do that typically.

The code for Sections 4 and 5 of GUI was a little simpler. I changed the font of the text displayed in the label widget of the output section and added a single button widget to allow users to exit the program:

How, having looked at all that code, let’s take a look at the GUI as it appears to the user:

Python GUI

Admittedly, not the most beautiful of GUIs ever to grace the screen of a monitor. Also, not the worst. My intention was never to make the GUI look sleek, rather to learn the basics of TKinter and apply them to a working project.

Functions

With the GUI complete, the next step was to code up the functions that would be triggered using the input and button widgets in the GUI.

Function 1

Role: to accept the selected level of difficulty and pass the value to function 2.

Here, the value of the variable ‘q’ is set by the button the user selects. If ‘Easy’ is selected, the integer 6 is stored in the ‘q’ variable and then passed to Function 2. Likewise, if ‘Medium’ or ‘Hard’ are selected, their respective integers are stored in ‘q’ and passed to Function 2.

Whichever difficulty is selected, a message appears on Section 4 of the GUI, advising the user of the chosen difficulty level.

Function 2 

Role: to accept user input, formulate a URL from the input, use BeautifulSoup to scrape the URL, randomly select a track, establish the name of the track, pull lyrics from the selected track, take the value from Function 1 and , enunciate the finished string of lyrics.

I am fully aware that this function is too long and should be broken into multiple functions, but it does the intended job. For readability, let’s do this in chunks. This first section of Function 2 takes the user’s band/artist info from Section 2 button. It takes that data and formulates a complete URL that is then fed to requests and BeautifulSoup to scrape the band/artist’s page for all existing albums.

Function 2.1

Next the function extracts all track URLs from each album and puts them in a list, from which a single track is selected at random. The function proceeds to establish the name of the selected track and stores it as a global keyword to make the name available to Function 3.

Function 2.2

And the next step is for Python to open the randomly selected track and extract the lyrics. As AZLyrics is a site that relies on user submitted entries, it’s often the case that formatting differs from artist to artist and even between track lyrics on the same album. For that reason, this part of Function 2 deals with cleaning up the extracted lyrics and removing unnecessary  characters that the text to voice won’t be able to deal with.

Function 2.3

The final role for Function 2 is to create a string based on lyrics from the randomly selected track and to pass that string to the robot voice for enunciation.

You will also note that I included a bit of error handling in Function 2 to make sure nothing is derailed if the user enters anything that problematic. Like misspelling an artists name or enter nonsense. Under such circumstances, an error message is displayed in  Section 4 of the GUI.

Function 3

Role: to accept user input, compare the guessed track name with the actual track name passed from Function 2, prepare a response based on correct/incorrect response, enunciate the response and then display the response on the text widget.

The code for this function can be split into two parts – the first part that composes either an insult or compliment, based on whether the user input is the same as the track name established in passed down from Function 2:

The second part is the code that makes the comparison and then enunciates the appropriate insult/compliment output:

When conceptualizing this program, I though that the text to voice aspect would be the most challenging part. However, thanks to the awesome Python text to speech library pyttsx3 , it was one of the easiest. Once the properties (voice gender and talking speed) were set, it was simply a matter of passing the desired text to the voice engine.

Next Steps

Although this project couldn’t be called 100%, as I took the core idea from the dev.to post, I was satisfied that I had made several improvements to the initial idea and increased the complexity with the GUI and user interface. As a first project, I am happy with the results and my daughter was thrilled to test whether she was a true fan of several of her favourite artists. That alone made it a worthwhile endeavor.