Posts

DSC_0024 Zach Doty Cover Photo for What is Machine Learning

What is Machine Learning?

Hello there, fellow Machine Learning (ML) students! Welcome back to our crash course in starting machine learning from an absolute beginner’s perspective.

In our previous article, we covered an introduction to Machine Learning, answering several key questions:

  • Where is machine learning used in our lives?
  • Where did machine learning come from?
  • Where is machine learning headed?

Forging ahead in our learning journey, we’ll introduce some definitions of machine learning and look at the major types of machine learning applications.

 

Machine Learning (ML), A Casual Definition by Arthur Samuel

Our first definition, teased in the last article, follows:

Machine learning is the practice of giving computers the ability to learn without being explicitly programmed to do so.

 

More on Arthur Samuel & Why His Definition on ML Matters

If you’re like me, you might not have heard of Arthur Samuel. Who is he, and why does his opinion matter in the fields of artificial intelligence and machine learning?

Arthur Samuel was a pioneer in artificial intelligence and computer gaming fields. In 1959, he coined the term “machine learning” as a founding father in the field. That’s why he’s important! Let’s also look at a more formal / scientific definition.

 

A More Formal Machine Learning Definition

Tom Mitchell, of Carnegie Mellon, offers a definition with more structure.

  • A well defined learning problem follows
  • E * T = P
    • Note: His definition does not include mathematical operators. I’m taking a large liberty to insert them myself. ¯\_(ツ)_/¯
  • Experience (E) placed against Task (T) is measured by Performance (P)

2017-04-06-001-Machine-Learning-Definition-ETP-Framework

Here’s a further example:

Example: playing Go.

E = the experience of playing many games of Go.

T = the task of playing and winning Go.

P = the probability that the program will win the next game.

 

Major Categories of Machine Learning Algorithms

If you judge by press coverage of ML as I have, it appears to be a nebulous field. (In all fairness, it may still be.) However, there is structure we can take in learning ML. There are two types of machine learning algorithms:

  • Supervised learning algorithms
  • Unsupervised learning algorithms

There are a couple of other prominent types of machine learning algorithms as well: reinforcement learning and recommender systems.

 

 Wrap-Up

Congratulations, we’ve cleared a very gentle introduction to machine learning, and it’s novice/high level definitions. I look forward to learning more with you, dear reader! Our next articles will cover a bit more detail about the two major ML algorithm types: supervised learning, and unsupervised learning. Until then, look after each other.

DSC_0013 Zach Doty Cover Photo for Introduction to Machine Learning

An Introduction to Machine Learning

Howdy! This is an abrupt interruption to our regularly scheduled programming of SQL lessons, Amazon Alexa Skill development and Algorithmic trading. For those readers who don’t personally know me, I’m on a quest/streak to level up as a technologist.

Getting around to the point, I’ve been taking self-paced courses in varying forms to learn, apply and share new skills. However, I’ve heard the Coursera Stanford class in Machine Learning taught by Andrew Ng recommended so widely, I’m just going to doggedly sprint a marathon. I’m starting almost a week behind, working a busy job, trying to have a social life, and many other things…but darnit, I’m going to give this class my best shot. Hopefully I finish. 🙂

Okay, over-sharing complete. Let’s jump into a brief introduction of Machine Learning.

Machine learning, according to Andrew Ng (Chief Scientist at Baidu), is the science of getting computers to learn without being explicitly programmed.

Where is machine learning used in our lives?

Machine learning is employed a large number of actors. Here are a few examples:

  • Search engines, such as Bing, use machine learning to process MASSIVE amounts of data to quickly rank web pages in order of relevance, with limited human intervention.
  • Social networks, such as Facebook, use machine learning to recognize your friend’s faces for auto-tagging capabilities.
  • Email providers such as Apple Mail may employ spam filters that continuously learn to protect your inbox, your computer, and most importantly, your sanity.
  • Tech companies such as Amazon use natural language processing (NLP) to create conversational experiences and transactions with skills and services.
  • Entertainment companies such as Netflix use self-learning algorithms to recommend compelling new films and TV shows for those of you who binge watch

Where did machine learning come from?

Machine learning originated from a computer science field known as artificial intelligence. Long seen as a pipe dream from Star Trek (crass and careless reference, I know), machine learning is a practical and attainable segue to artificial intelligence, or machines and programs that contain some degree of self awareness.

This capability is a relatively new, yet a rapidly exploding field that grows as mathematical, statistical, hardware and software capabilities continue to compound and improve.

What follows is a better question still.

Where is machine learning headed?

Machine Learning in the future could look like a few different things (but not limited to this list, obviously!):

  • Predictive and preventative applications in engineering, medicine, and security
  • “Load bearing” performance in complex tasks, such as architecting, coding and programming self-driving cars
  • Coordinate machines and programs that study our behavior at our request and perform tasks, such as performing spring cleaning
  • Assistants or programs that are intelligent – able to optimize and independently solve problems on our behalf

Wrap-Up on the Machine Learning Introduction

That wasn’t so bad was it? We’ll follow soon with a more formal definition of Machine Learning and its various tranches of study. Cheers all.

DSC_0104 Zach Doty Cover Photo for Learning LIKE SQL Statement

The LIKE Statement: SQL Statement Fundamentals

Howdy folks! We are overdue for another installation of SQL learning. I’ve slept a few times since the past couple of articles on how to learn SQL. Previously, we talked about the IN Statement, BETWEEN statement, and ORDER BY clause.

In this article, we’ll learn how to execute the LIKE statement in SQL queries. Let’s jump in.

 

About the LIKE Statement & Why it’s Important

Have you ever worked with a data set that’s overwhelmingly large  or complex? Or overwhelmingly large and complex? Sometimes, you need to find data, but can’t recall the exact string or values for a lookup. Or, perhaps, you’re working with a messy data output from say, Google Keyword Planner that groups a range of close variants into one value?

Say you’re looking for values related to designer clothing and designer clothes. Without a better solution, the most probable solution for you is to do a bunch of sorting, filtering, classifying and other data sleuthing at great expense to time and sanity.

The LIKE statement exists to help with the debacle of only having / knowing  part of the lookup criteria you need, courtesy of pattern matching.

PostgreSQL and many other SQL engines/platforms support the LIKE statement, which functions a bit like the below. (Pun not intended.)

 

SELECT keyword, search volume

FROM table

WHERE keyword LIKE ‘cloth%’

 

The above tells pgAdmin / PostgreSQL to get the keyword and search volume columns from table, where the keyword values match values that begin with ‘cloth’ and are followed by anything else, the percent sign. The combination of calling out a text string with an operate is known as a pattern.

When you execute the LIKE statement in a SQL query, pgAdmin will begin reading through the table rows to see if the pattern you’ve specified returns any matches. For the season marketing technology folks, this functionality sure does resemble regex in some ways.

However, there are some differences.

  • Here, instead of the * character being wildcard, the % sign serves as a wildcard matching all characters.
  • If you want to match a single character, the underscore character is used.

 

LIKE Statement Syntax & Examples

Let’s try some examples. Below, we’ll call on the faithful DVD rental practice database, and run a query for customers that have first names like Jen. (Jennifer, Jenny, etc.) Our below code produces the following result.

SELECT first_name,last_name

FROM customer

WHERE first_name LIKE ‘Jen%’;

2017-03-30-001-LIKE-SQL-Statement-Example

 

That being considered, there are other ways we can use the like statement. Above, we used a wildcard to match any endings to a particular string.

Conversely, we could execute a SQL query that specifies a certain ending value, with the wildcard preceding. If we extend that example to such a query below, we should see the following result:

SELECT first_name,last_name

FROM customer

WHERE first_name LIKE ‘%y’;

2017-03-30-002-LIKE-SQL-Statement-Example

 

Above, we’ve flipped the tables so we capture every possible beginning condition under this pattern. Also important to note, the patterns aren’t limited beginnings or ends. You can use this wildcard in the middle, etc. Consider the following:

SELECT first_name,last_name

FROM customer

WHERE first_name LIKE ‘%er%’;

2017-03-30-003-LIKE-SQL-Statement-Example

 

Now, on top of all this awesomeness, realize we’ve been using it as a matching filter. We can also employ the NOT LIKE syntax to exclude values meeting the pattern we’ve specified.

Let’s mix things up a bit. We mentioned a second type of pattern matching character that we haven’t used yet: the underscore.

SELECT first_name,last_name

FROM customer

WHERE first_name LIKE ‘_her%’;

2017-03-30-004-LIKE-SQL-Statement-Example

 

Above, we’ve made a similar ask. However, instead of requesting all possible matches, we’ve mandated SQL only return the ‘er’ string that begins with ‘h’.

To throw you a quick curveball, it’s worth noting the LIKE statement is case sensitive in its matching. Could be bad, could be good, could be neither. However, there’s a way for you to force case insensitivy on the queries. The difference in your statement appears relatively minor. Instead of using a function that calls LIKE or NOT LIKE, you’ll use ILIKE.

 

Wrap-Up

Hope you found this useful! Stay tuned for more SQL learnings and application. If you’re new here, visit the page on how to learn SQL. If you’re interested in more educational material, check out our ongoing series of how to develop Amazon Alexa voice search skills, and getting started with algorithmic trading. Cheers!

Building an Interactive Alexa Quiz Skill, Part 2

Disclaimer: this was typed late at night on a tired mind. Please excuse typos, convention errors, and generally poor writing. 🙂

Howdy, Alexa nerds! Welcome back to our journey in learning Amazon Alexa Skill Development. Quick funny aside, would you care to guess my most common use of the Echo? It’s to play looong Spotify playlists that are basically background noise to help our new dog when Hannah & go to the gym or meet friends. Anyway!

Let’s jump back in. The previous article covered setting up an AWS Lambda function for the Alexa Skill Service. Now, we’ll be working more with the Skill interface. See below for the conceptual overview, or an early article on building Alexa skill interfaces for a basic fact skill.

2017-03-07-001-Alexa-Skill-Interface-Develpoment-Framework

 

Working in the Amazon Developer Console: Alexa Skills Kit

You probably know the drill now, log in to the Amazon Developer console. Once you’ve logged in, select the “Alexa” menu item from the home screen, then choose the “Alexa Skills Kit” Option.

2017-03-27-B-001-Alexa-Quiz-Skill-Create-Alexa-Skill-Interface-Start

If you’ve previously published or started development of skills, you should see them listed on this screen. Now, click, “Add a New Skill”. We should be looking at a very familiar screen here. 🙂

Add/edit the following:

  • Language (assuming you’ll leave the English US default here)
  • Name of the Skill displayed in the Alexa app and store
  • Invocation name users will speak to start your skill

Click Save and Next to proceed.

2017-03-27-B-002-Alexa-Quiz-Skill-Create-Alexa-Skill-Information

 

Working with the Interaction Model

Here comes the tough part, more copy and paste! Okay, sarcasm and humor doesn’t always translate well via text. We’re going to continue to lean fairly heavily on Amazon’s examples here to get ourselves familiarized with the more advanced concepts of intent schema and slot types.

That caveat aside, head back to the files we originally downloaded, but this time, we’re interested in the speechAssets folder and its contents:

  • json
  • Sample utterances (text document)

First, let’s open up the Intent Schema JSON file in our text editor of choice. Below, a look at what you should approximately be seeing. Copy and paste the entirety of the JSON file into the Intent Schema field of the Interaction Model tab.

2017-03-27-B-003-Alexa-Quiz-Skill-Create-Alexa-Skill-Intent-Schema-JSON

Audible: Our First Encounter with Custom Slot Types

Alright, no smooth segue here. We’re having the first encounter with what’s known as custom slot types.

If you were to try and save the skill progress so far, you’ll receive an error message from the developer console that says something like, “Error: There was a problem with your request: Unknown slot type ‘LIST_OF_ANSWERS’ for slot ‘Answer’. Why is that?

2017-03-27-B-004-Alexa-Quiz-Skill-Create-Alexa-Skill-Custom-Slot

If you take a closer look at the Intent Schema JSON file, you’ll notice that  most of the intents are built-in Amazon intents. E.g., “intent”: “AMAZON.RepeatIntent”. The “AnswerIntent” looks nothing like the built-in Amazon intents. Instead, we see a name, “Answer” and type, “LIST_OF_ANSWERS” that was so delicately referenced in the error message.

So how do we remedy this situation? We use the information presented to us in the error message and the JSON file to work our way over this issue. You’ll likely note under the custom slot types mentions “Enter Type”.

2017-03-27-B-005-Alexa-Quiz-Skill-Create-Alexa-Skill-Custom-Slot-Values

Match that information up with our error message and the JSON code, and we’ll enter, “LIST_OF_ANSWERS”. In the values section, we’ll enter on separate lines: 1, 2, 3, and 4. I’ll note here for clarity, that this essentially corresponds to the A/B/C/D multiple choice functionality of the quiz. We’ll see this in greater detail in a bit.

Okay, click “Add” as highlighted above, then click “Save”. Next, return to your files and open up the Sample Utterances text file. You should see something like the below.

2017-03-27-B-006-Alexa-Quiz-Skill-Create-Alexa-Skill-Sample-Utterances

You’ll note it’s quite a bit different than the previous, simple, fact-dispensing Skill we previously built. Take note of the {Answer} sample utterance. These are the pieces of dynamic input and interaction coming together into an Alexa Skill. We’ve defined a custom interaction outside of Amazon’s standard functions, and specified a range of acceptable answers the user can give us. That whole structure meets the user experience here, called in by the {Answer} slot name and custom slot type.

Enough conceptual babble. Copy and paste the sample utterances text into the developer console! Click Save beneath the sample utterances, and click Next.

2017-03-27-B-007-Alexa-Quiz-Skill-Create-Alexa-Skill-Sample-Utterances-List

 

Continuing the Skill Interface Build-out

Alright, so far, we’ve accomplished the following:

  • Provided basic skill information about our new skill
  • Specified details about the interaction model, including;
    • Intent Schema
    • Custom Slot Type
    • Acceptable/specified values for the custom slot type
    • Sample utterances

Next, we need to fill in some simple but crucial configuration details. Remember the ARN we generated by setting up the AWS Lambda function in the previous article? You need it here. Below you can see:

  1. I’ve selected the recommended endpoint type of AWS Lambda ARN
  2. Selected my geographic region of North America and,
  3. Pasted in the full ARN

I’m not going to work with account linking yet, because honestly, it looks really darn complicated and its well past midnight as I type this. Soon. 🙂 Click next and proceed to the testing tab!

In the testing tab, you should first see that the skill is enabled for testing on your account. You can:

  • play back responses from Alexa in the voice simulator to test pronunciation, etc
  • More importantly, use the service simulator to run a sample utterance, and see if your skill actually works.

2017-03-27-B-008-Alexa-Quiz-Skill-Testing-Alexa-Skill

Above, we can see the response to our sample utterance asking SEO Quiz returns as expected. Woohoo! Also, did you know the Alexa voice simulator automatically bleeps out most curse words? Did you know you can kind of work around that by putting extra vowels in the word? I digress. (It’s almost 1 am writing this now, productivity on the rise!) When you’re satisfied, click Next.

We’re getting close! Time to enter some publishing information. I’ll leave the first few sections to you: Category, Sub-Category, Testing Instructions, Country/Region availability, Short and Full skill descriptions.

Now, in the example phrases, I provided some updates to the sample utterances, namely to the starting Intent. Below, see the example phrases of “Alexa open SEO Quiz” and so forth. The “gotcha” here that set me back on my first skill is that the example phrases must be derived from your sample utterances.

2017-03-27-B-009-Alexa-Quiz-Skill-Testing-Alexa-Skill

Upload your 108×108 and 512×512 pixel icon images, click Next and submit the requisite privacy & compliance information. Done!

 

Wrap-Up

So, we’re mostly done, not completely done. The part for usto do now is customizing the template code in your AWS index.js file. Ideally, I would prefer a more eloquent closing, but it’s late, will have to wait for another time. Look after each other.

 

DSC_0002 Zach Doty Cover Photo for Interactive Alexa Quiz Skill Development

Building an Interactive Quiz Alexa Skill, Part 1

Hello Alexa geeks! Welcome back to our journey of learning how to develop Amazon Alexa Skills for the Echo and more. Last time, we completed the build process for our first simple “fact-dispensing” Alexa Skill.

In this article, we’ll start the process for a skill that accepts user input in the form of a quiz, fun! If you recall from our first skill, there are two parts to skill development:

  1. The Skill service development, in AWS (Lambda)
  2. The Alexa Skill interface details through the Amazon / Alexa Developer Console

 

2017-03-01-001-Alexa-Skill-Develpoment-Framework

 

Getting Started in AWS Lambda

You’ll notice as we progress from our early articles, there will be less detail paid to more basic instructions, such as our first! First, log in to the Amazon AWS portal.

Navigate to the Lambda service. If you’re the casual developer just working in this course, odds are the Lambda link will be near the top of screen under “Recently Visited Services”. Once you’ve clicked through, click, “Create a Lambda Function”.

2017-03-27-001-Alexa-Quiz-Skill-Create-AWS-Lambda-Function

On the next screen, you should see something like “Select blueprint” (Note: at the rate of change Amazon has been pursuing, this screen could change, even in a matter of weeks!) Click the “Blank Function” option, we’re starting this one from scratch!

2017-03-27-002-Alexa-Quiz-Skill-Create-Blank-AWS-Lambda-Function

The next screen should be, “Configure triggers”. Click inside the gray dash-outlined box, and select, “Alexa Skills Kit” from the dropdown menu. Click next!

2017-03-27-003-Alexa-Quiz-Skill-Configure-Alexa-Skills-Kit-Trigger

 

AWS Lambda Function Configuration for Alexa

Now we should be able to configure the basics of our function. Enter the following:

  • Function name
  • Description

The default runtime environment should be Node.js 4.3. If not, change it to Node.js 4.3.

(Note: Amazon just introduced support for Node.js 6.10, so that may be the preferred format going forward!) Will try to provide an article update, should that be the case.

2017-03-27-004-Alexa-Quiz-Skill-Configure-Alexa-Function

Onward! Now, we need to upload some code to this burgeoning success. Throwback time, do you remember the files we downloaded in one of the first articles? Time to go back to them again. In your folder of numbered skill templates, go to “2-reindeerGames”, “src” folder and safely open the index.js file in your text editor of choice.

2017-03-27-005-Alexa-Quiz-Skill-Lambda-Function-Code

Copy and paste (replacing all previous code) into the code window that should appear.  This assumes you’ve selected the Code entry type of “Edit code inline” for the Lambda function code. As we work on more increasingly more advanced skills, we will likely use the zip upload feature to accommodate additional code resources. The astute will note we’ve merely copied and pasted code here. Yes, we’ll go back and customize soon. 🙂

Beneath the code window, leave the index.handler intact, select an existing role option in the Role dropdown menu, and use the role we previously created. Leave the other settings as-is, click the “Next” button to review details, and click, “Create Function”!

2017-03-27-006-Alexa-Quiz-Skill-Lambda-Function-Creation

Be sure you take note / record the ARN in the upper right-hand corner, as we’ll need that in our forthcoming Skill Interface development section.

Wrap-Up

That’s the first part! I don’t know about you, but this is getting easier as I go. We’ll next cover the skill interface and customization to make it your own skill. If this is your first article, be sure to check out the running stable of articles on how to learn Amazon Alexa skill development. Also, there’s a growing body of work on how to learn PostgreSQL, and some fledgling articles on learning algorithmic trading, for good measure.

Share your experience, thoughts and feedback in the comments below. Don’t be a stranger, help your friends along in Alexa Skill development and share with them. Cheers!

 

 

 

 

 

 

How to Build Your First Amazon Alexa Skill, Pt. 2: Skill Interface

Welcome back, Alexa geeks! In the last article, we laid the groundwork for making our first Amazon Alexa skill. We covered the concepts and frameworks in Alexa skill development. We also did some work in AWS Lambda to prepare for voice requests  being made to our service.

 

Quick Recap on The Alexa Skill Service / AWS

We’ve slept a few times since covering the concepts and AWS framework, so let’s quickly recap.

2017-03-01-002-Alexa-Skill-Process-Framework

Whenever we use an Alexa skill , our voice data is processed through the hardware device, through the skill interface (what we’re looking at today) for language processing, then converted into text for a program to execute against, and back again. Simple enough, right?

In the last article, we looked at the “last” leg of this process, the skill service and AWS Lambda. Now, we’ll be working with that and the skill interface portion.

2017-03-07-001-Alexa-Skill-Interface-Develpoment-Framework

 

Setting Up the Basic Skill Information

Alright, log into the Amazon Developer Portal and Select the Alexa menu option. Once there, you should have a choice between the Alexa Skills Kit or the Alexa Voice Service. Click on the Alexa Skills Kit link to continue. You’ll want to click, “Add a New Skill”. Because I’ve already developed a skill, Silly Marketing Strategies is already there. However,  most of y’all will probably not have anything else on screen.

2017-03-07-002-Alexa-Skill-Interface-Develpoment-Framework

To start, you’ll want to select Custom Interaction Model, leave the default language to US (unless you’re developing in Espanol?). Type out the name and invocation name for your skill. The name isn’t necessarily important this moment. However, the invocation name will be extremely important!! This will be how users will call your Skill into service.

Note: the astute will notice I’ve deviated from the Space Geek example of the last article. More on that later. 🙂

When you’re ready, click next to proceed, and we’ll tackle the interaction model.

2017-03-07-003-Alexa-Skill-Interface-Develpoment-Framework

 

Setting up the Skill’s Interaction Model

Alright, time for some important things here. For the purposes of the factoid-based game, we’ll only be looking at the Intent Schema and Sample Utterances fields.

First, we need to look at the Intent Schema. Remember the files you downloaded in the first article? You’ll need to go into the SpaceGeek folder, Speech Assets sub folder and open the IntentSchema.json file. I’ve opened it up in Sublime Text 2 briefly, so we can take a quick look at the file. So this is JSON, with intent pairs. Below, we’ve got a pretty simple set, intents for retrieving a fact from the skill, getting help, stopping and cancelling a function. It’s easy right? Ha.

2017-03-07-004-Alexa-Skill-Interface-Develpoment-Intent-Schemas

Quick note: because these intents are proceeded by Amazon – it means they’re built in for Amazon. Enough babbling, copy and paste the contents of this file into the Intent Schema section of Amazon Developer Console.

2017-03-15-005-Alexa-Skill-Interface-Develpoment-Intent-Schemas-Utterances

Above, we’ve pasted in the Intent Schema. Next, we need to provide some sample utterances. Sample utterances are what you think users might say to engage your Alexa Skill. Below, we’ve provided such examples as, “Tell me a Weimaraner fact”.

Next, we need to hook up the Alexa Skill interface we’ve put together with some computing power. Specifically, we need to hook it up to AWS Lambda! (Remember the first article where we did a bunch of Lambda setup?)

If you recall, there was an Amazon Resource Name (ARN) string that we copied and saved to a text file. Retrieve it now and paste into the “Configuration” screen of the Skill interface setup.

2017-03-15-006-Alexa-Skill-Interface-Develpoment-Intent-Schemas-Utterances

Providing the ARN you’ve provided is valid, you should be able to proceed to the next step. Note: we are ignoring the account linking functionality for now. This functionality allows you, for example, to integrate Twitter sharing functionality into your skill by sharing a Tweet.

Next, we’ll move on to the Test tab. Three things (below) to take note of:

  1. Ensure you’ve completed the Interaction Model tab, so you can complete the testing in this tab.
  2. Try / type out key phrases in your skill to hear how they’ll be pronounced, via the Voice Simulator.
  3. Enter some utterances into the Service Simulator to A) make sure your skill is functioning as intended and B) Get a feel for how the end user experience will happen

2017-03-18-006-Alexa-Skill-Interface-Develpoment-Testing-Tab

Once you’ve tested your Skill, proceed to Publishing Information. Here, you’ll need to include the following:

  • Category of your Alexa SKill
  • The relevant Sub Category
  • Optional: testing instructions if your skill requires credentials or other unusual needs. You probably don’t need to include anything for this example
  • Country & Region targeting
  • Short Skill Description
  • Full Skill Description
  • Example Phrases, drawn from your sample utterances, and preceded by the Alexa wake word and skill invocation name
  • Optional: keywords that will help Alexa users find your skill in search
  • Images in 108 and 512 pixel squared dimensions

Below are a couple of screenshots for how I’ve filled out these fields.

2017-03-18-007-Alexa-Skill-Interface-Develpoment-Publishing-Information

2017-03-18-008-Alexa-Skill-Interface-Develpoment-Publishing-Information

Alright, we’re so close! The last field is Privacy & Compliance. For this example, you should be checking “No” to all the radio buttons:

  • No, skill doesn’t allow users to make purchases or spend real money
  • No, skill does not collect personal information from users
  • No, skill does not target children under the age of 13

However, do note that some of these things (except for child targeting) may change as we progress in our Alexa Skill development capabilities.

If desired (or later required by more advanced capabilities) you may specify privacy policy and terms of use URLs.

If you’ve completed the above, you should be good to Save & Submit for Certification!

 

Wrap Up

This is a deceivingly involved process. You will note that neither in the previous article, nor this article, have we changed the original source code for the SpaceGeek / Weimaraner Facts Skill. We’ll cover this in more detail in the next article. Until next time, check out my journey in learning NoSQL and keep an eye out for more content soon!

 

The IN Statement: SQL Fundamentals

Howdy, budding SQL masters! It’s been a minute since the last SQL post! I’ve taken some time to devote to Alexa Skills, some other personal housekeeping, and adopting a dog with my wife!!

Last time we talked SQL, we learned the BETWEEN statement to hone in on the exact values we wanted for a SQL query. Prior to that, we looked at the LIMIT statement to get a small return data sample and also working with ORDER BY to take sorting and filtering into our own hands.

Today we’ll be working with the IN statement.

About the IN Statement

The IN SQL operator is a companion to the WHERE clause to see if a value matches any value in a list of values. Sounds something kind of like VLOOKUP’s, yeah? The syntax of an IN statement might look like:

value IN (value1, value2)

The expression in your SQL query will return true if the value(s) you’ve specified in the query match any value in the list you’re referencing.

Sub-queries: Using the IN Statement with SELECT

The list of values is not limited to a static number or list of strings. That is, you can be a bit more dynamic and free-flowing with your query by getting the value via a SELECT statement.

This is also known as a sub-query. To illustrate, the syntax might look like:

value IN (SELECT value FROM table)

NOT IN: Not Making Fetch Happen

The IN statement has a similar corollary as the BETWEEN statement- a NOT modifier. So, we can take the inverse of a statement. (Drawing on some experience in data analysis, this can just as, if not more useful than the original statement!)

value NOT IN (SELECT value FROM table)

 

Sample SQL Queries for the NOT IN Statement

Alright, let’s run some sample queries using the IN statement! Here’s our first go, working again with the ever-present dvdrental sample database. Below, we are selecting rental information for customers matching only a certain ID cohort (think perhaps loyalty group?), with a few columns and sorting by descending order for the return date:

SELECT customer_id,rental_id,return_date

FROM rental

WHERE customer_id IN (1,2)

ORDER BY return_date DESC;

Below, we see our results returned as intended!

2017-03-13-001-IN-SQL-Statement-Example 

How about our corollary? If we want to add the NOT IN modifier, it’s as straightforward as you would imagine.

FROM rental

WHERE customer_id NOT IN (1,2)

ORDER BY return_date DESC;

2017-03-13-002-IN-SQL-Statement-Example-2

Above, we again receive the desired result. I believe we also get a lesson in the importance of clean data here, as blank values are presented first. Let’s try one more sample query, just for kicks. We’ll switch it up ever so slightly, now working with the payment table.

SELECT city_id,city,last_update

FROM city

WHERE country_id IN (44,82,60)

ORDER BY city ASC;

2017-03-13-002-NOT-IN-SQL-Statement-Example

Above, we’ve demonstrated that we don’t necessarily need to display a column we use in the query (country_id) and have sorted by city name in ascending order.

 

Wrap Up

So why the IN statement? At face value, it seems rather simplistic among SQL clauses we’ve explored thus far. Let’s consider the following.

The IN statement allows us to avoid a messy swath of values courtesy of BETWEEN (if we’re working with a proper large data set), or daisy chain list of a bunch of equals OR statements. Here’s another plus, it’s thought by some that pgAdmin / PostgreSQL will execute the IN query faster than the list of OR statements.

Alright, that’s it for today’s article. If you’re like me and need a quick refresh, revisit our page on Learning SQL that contains all the topics we’ve covered so far.

 

DSC_0005 Cover Photo by Zach Doty for Build Your Own Amazon Alexa x Echo from Raspberry Pi

How to Build Your Own Amazon Alexa with a Raspberry Pi

Howdy, it’s been a minute since my last post on Alexa Skill Development. I left us hanging, as I mentioned in one of my previous SQL posts, work craziness, sickness, and general life stuff happening imposed a brief, involuntary hiatus on writing.

Enough self talk to make future self feel better. Let’s build our own Amazon Alexa with a Raspberry Pi! What a step up from our previous articles. 🙂

But first! A couple of notes: This is NOT necessary for you to do in learning Alexa Skill Development. You can use a regular Echo instead of this expedition into Computer Homebrew 102. Also, I developed my first Alexa skill without an Echo or comparable device at all. One more thing, this is a long post that is both deep and wide ranging. Don’t say I didn’t warn you.

1. The Materials List

To get started, you’ll need a few things:

  1. Raspberry Pi, version 2 or later
  2. Power source
  3. Micro SD card (Recommended at least 8 gb in storage size)
  4. Speaker with a line-in 3.5mm cable (or headphones of similar line-in spec)
  5. USB microphone
  6. Keyboard and Mouse
  7. HDMI cable to connect to a monitor or TV
    1. Obviously, you need a monitor or TV. 🙂

 

A few notes on the Raspberry Pi: I recommend you get a pre-assembled kit, for time and convenience, if nothing else. I previously purchased from CanaKit on Amazon, and have been pleased with it. If, for some reason, you are building your own or need to reformat, the two articles below from RaspberryPi.org are simple, fast instructions to get you up to speed.

 

Formatting a MicroSD card 32 GB or less:

https://www.raspberrypi.org/documentation/installation/noobs.md

Downloading New Out of Box Software (NOOBS) for Raspberry Pi, in preparation of your first boot.

https://www.raspberrypi.org/downloads/noobs/

 

Okay, we’re going to proceed assuming that you’ve taken care of the above, and have successfully booted your Pi, connected all requisite hardware, connected to the internet, and so on. Below is where I’m at.

2017-02-20-001-Alexa-Raspberry-Pi-Successful-Boot

2. Installing Other Utilities

Another note here, it is possible to SSH (read: a bit like Microsoft’s RDP remote desktop application, but command line style) your way into the Raspberry Pi. I will not be doing that. This may sound a bit silly, but I don’t have a multi-screen dock for my laptop (capped at 2), and I’m happy for the Pi and other monitor to be a standalone computer. (Perks of a wide desk.) Also, the Wi-Fi USB card I’m using is very slow at the moment. Also, an edit after I’ve written this post offline: Linux-esque command line work is a royal pain in the you-know-what. Remarkable computing has advanced to where it is today. More on Raspberry Pi intricacies later.

Enough of my discourse, let’s continue setting up our Raspberry Pi / DIY Alexa to-be. Task time.

Navigate to your Pi’s command line module.

  1. We’re going to update the GET. Type:
    1. sudo apt-get update (Pro tip: don’t misspell sudo as I’ve laughingly done on the first try.)
    2. In the new line created by default, type: clear and hit enter. This will clear your command line.

2017-02-20-003-Alexa-Raspberry-Pi-Get-Update

3. Registering Your Raspberry Pi as an Alexa Voice Service Device Type

Alright, head to the Amazon Developer Portal and log in with your credentials.. (If you’ve followed along with previous posts, you have a login! If not, see this article on setting up an AWS account.)

C-2017-02-23-001-Alexa-Raspberry-Pi-Registration

Once you’ve logged in, click on the Alexa navigation item, then click “Get Started”. You should land on a page that has a “Register Product Type” drop-down. Select Device and continue.

C-2017-02-23-002-Alexa-Raspberry-Pi-Registration

You’ll be taken to a new screen / process to create a new device type. Choose and type your  device type ID and display name. For ease of process here, I’ve used the same value for device type ID and display name.

C-2017-02-23-003-Alexa-Raspberry-Pi-Registration

Once you’ve selected your ID and name of choice, click next. Now we need to create a new security profile. Click the dropdown menu that reads, “Select Security Profile” and click, “Create New Profile”. If you’ve filled out all the fields properly, you should have the option to submit this device / product. Hooray! Below is what I see after I submit the new device.

C-2017-02-23-004-Alexa-Raspberry-Pi-Registration

Create a Security Profile Name and Description of your choosing and click “Next”.

C-2017-02-23-005-Alexa-Raspberry-Pi-Registration

You’ll now see a new screen with your created security profile. Below, you should see some tabs such as, “General”, “Web Settings”, “Android/Kindle Settings”, and so forth. Click on “Web Settings” and then click “Edit”.

C-2017-02-23-006-Alexa-Raspberry-Pi-Registration

You should now be able to edit the “Allowed Origins” and “Allowed Return URLs” fields. Enter the following values into each of the fields, respectively:

1) https://localhost:3000

2) https://localhost:3000authresponse

Once you’ve entered the above values, click “Next”.

C-2017-02-23-007-Alexa-Raspberry-Pi-Registration

Now we need to add some additional detail about our device type.

  • Upload an image (I used this one, they’re picky about sizing)
  • Choose a category (I selected “Other”)
  • Provide a short description to your pleasing
  • Select “No” for plans on making product available to general public
  • Select “No” for product directed to children

C-2017-02-23-008-Alexa-Raspberry-Pi-Registration

Click “Next” to proceed. The next screen asks about applying for access to Amazon Music. I’m selecting “No” for the purposes of this exercise.

4. Enable Login With Amazon for your new Device / Product

Sweet! After you click, “Submit”, you should see the below. Now head over to the “Apps & Services” section, and select the “Login with Amazon” item, once the Apps & Services screen has loaded.

C-2017-02-23-009-Alexa-Raspberry-Pi-Registration

From the drop-down menu of existing security profiles, select the profile we just created. You’ll need to enter a consent privacy notice URL and a consent logo image. Because we’re doing anything public facing, you can enter any URL you’d like here. For the image, I used the same image from the device creation process. Click “Save” and let’s keep moving.

C-2017-02-23-011-Alexa-Raspberry-Pi-Registration

Upon a successful save, you should see a table with a column labeled “Oauth2 Credentials”. Click, “Show Client ID and Client Secret”, copy and save the information somewhere safe. Next stop, Raspberry Pi.

5. Install the Application on Your Raspberry Pi

Flip back over to your Raspberry Pi and type the following two bullet point texts as unique lines into command line.

  • cd Desktop
  • git clone https://github.com/alexa/alexa-avs-sample-app.git

It should take a minute or two to process, and you should see the below.

C-2017-02-23-012-Alexa-Raspberry-Pi-Sample-App-Setup

Next, type the below point texts as unique lines into command line:

  • cd alexa-avs-sample-app/
  • nano automated_install.sh

After returning the two above lines, you should see something like the below. Remember what we entered in the Developer Console a few steps ago? You’ll need this information here. (Highlighted boxes.) Update from 2/25/2017: I actually had quite a bit of trouble with this step. Linux command line / text emulator is not my friend yet. The trouble I had here was making and saving my changes to the below. However, the program is surprisingly helpful! It will prompt you to verify credentials before install proceeds.

C-2017-02-23-014-Alexa-Raspberry-Pi-Sample-App-Setup

After you’ve made the appropriate changes, command line language of CTRL + X should help you save the changes. Close out the command line window and re-open. Type the following bullet texts:

  • cd Desktop
  • cd alexa-avs-sample-app
  • . automated_install.sh

When prompted for the AVS + Raspberry Pi License and Agreement, obviously answer and/or “y” to proceed, and answer yes and appropriate and desired throughout the setup process. E.g., you’ll be asked about language preference, audio input settings, enabling wake word detection, and so forth. (It takes anywhere from 30 min to 1 hour.)

Once the install runs, it’s time to start the Alexa service!

6. Running the Alexa Service / Finishing Touches

To test and run the Alexa service, we need 3 command line terminals open. The command paths for these are at the end of the install. (As I type this, the latest install on my Pi is approaching 45 minutes, so I’ll have to give you the bare bones for now based on past tests and some generous documentation.

The first terminal will set Alexa up to be listening on port 3000 (remember the local host URLs from earlier?) The second window deals with setting up a Java client and logging into Amazon with the Security profile we set up. Logging in and confirming will enable you to initiate the connection to Alexa, paving the way for the third terminal, which enables the wake word detection and actual running of the Alexa service. Woohoo! I’ll update with examples later.

Update — 2/25/2017:

So, the whole process took a looooong time. At about 3 am of the night I wrote / finished this post, the Pi finally finished unpacking and installing everything. Below is our success confirmation! I’ll be testing this evening…

Wrap-Up

Goodness, what a ride, if you stuck it through the whole article. You could do this, or just get a darn Echo. After all this, I recommend you buy an Echo. 🙂

Other miscellaneous updates from 2/25/2107: By the time I got to the wrap up, it was 3:30 in the morning, yikes! I wasn’t feeling very descriptive, nor was I yesterday either. This was an incredibly long process that took several days from start to finish. Was it worth it? Yes. I’m still going to test the capabilities tonight and hopefully have some follow up’s in the coming days. Will I be primarily using an Echo moving forward? Yes.

The BETWEEN Statement: SQL Statement Fundamentals

Hey there, folks! Welcome back to our journey in learning SQL. Our last few of posts covered the ORDER BY clause, LIMIT statement and the COUNT function. (See full list of SQL tutorials here.)

Today, we’re going to cover the BETWEEN statement. This is the start of some deeper material. In addition to the BETWEEN statement, we’ll also soon be covering IN and LIKE statements.

 

About the BETWEEN Statement

The BETWEEN statement (rather, operator) is used to match a value against a specified range of values. Maybe we want to get transactions between a certain dollar amount.

For example, value BETWEEN low AND high;

More about the BETWEEN statement. If the value is greater than or equal to the low value and  less than or equal to the high value, the expression returns true, or vice versa. Also, the BETWEEN operator can be rewritten by using the greater than or equal to ( >=) or less than or equal to ( <=) operators as the following statement.

value >= low and value <= high;

One other way to think about the BETWEEN statement is that takes two WHERE statements and lumps them into one. Think back to the WHERE statement, as we would say first:

SELECT column1 FROM table WHERE column1 >= 2 AND column1 <= 7;

 

To BETWEEN or NOT BETWEEN

Conversely, you can extend the usage of the original BETWEEN operator to NOT BETWEEN. This is similar in concept, except working toward exclusion, instead of inclusion. So, if we want to check if a given value is outside of a range, we can use the NOT BETWEEN operator as below.

value NOT BETWEEN low AND high;

Again, similarly to the double WHERE statements, NOT BETWEEN simplifies the burden into a single statement. For sanity, we’ll skip the SELECT WHERE example.

 

Using BETWEEN in pgAdmin / PostgreSQL

Below, we’ve taken the BETWEEN statement for a road test on our address table.

SELECT address_id,address FROM address

WHERE address_id BETWEEN 10 AND 20;

2017-02-18-001-BETWEEN-SQL-Statement-Example

How sweet was that? Syonara, comparison operators. Conversely, if we convert the above query into a NOT BETWEEN operator, we should see the below statement return the following results.

SELECT address_id,address FROM address

WHERE address_id NOT BETWEEN 10 AND 20;

2017-02-18-002-NOT-BETWEEN-SQL-Statement-Example

Okay, that was cool. Let’s hold up for a quick second. What about non-integer columns, such as data? Caveat: there’s a lot more to data types than our humble example below. For the purposes of this article, we can take a YYYY-MM-DD date and place the values into strings.

SELECT rental_id,rental_date,inventory_id FROM rental

WHERE rental_date BETWEEN ‘2005-05-24’ AND ‘2005-05-27’;

2017-02-18-003-BETWEEN-Non-Integer-SQL-Statement-Example

Voila!

Wrap-Up

Alright, for those of you reading this later, it’s 2 a.m. on a Friday night / Saturday morning, and I’m about ready to call it a night. Sorry I’m not providing a better conclusion for you today. I’m excited to delve further into more complex SQL queries. Keep track of my shared journey of a beginner learning SQL. Cheers!

 

 

DSC_0163 Zach Doty Cover Photo for ORDER BY SQL Clause

ORDER BY Clause: SQL Statement Fundamentals

Welcome back to our SQL learning journey! It’s been a week and a half since my last post on using LIMIT. Work got crazy and I got the flu! 🙁 But we’re back in action. Today, we’ll delving deeper into SQL statements in PostgreSQL: ORDER BY.

Let’s jump in. Why would you need a statement that orders data? Whenever you query data from a table, PostgreSQL will by default return the rows in the order they were inserted into the table. (Read: not the order you want.)

To sort the result set from your SQL query, you can use the ORDER BY clause in the SELECT statement and specify a certain ascending or descending order.

Sample ORDER BY Clause Syntax

Here’s an example of the ORDER BY clause would look like within a SELECT statement.

SELECT column_1, column_2

FROM table_name

ORDER BY column_1 ASC / DESC;

Some important notes for the ORDER BY clause:

  • Specify the column you want to sort by with the ORDER BY clause
    • If you sort the results by multiple columns, use a comma to separate between the two columns
  • Use ASC to sort the results in ascending order
  • Use DESC to sort the results in descending order
  • Should you leave the ORDER BY clause blank, it will use ASC (ascending) by default

ORDER BY Clause Examples in SQL Statements

Alright, first up. We’ll do a basic SELECT statement, adding both ORDER BY and LIMIT clauses. Code and screen shot below. The query should select the first and last name columns from the customer table, ordered by last name ascending (A > Z), returning only the first 10 rows.

SELECT first_name,last_name  FROM customer

ORDER BY last_name ASC

LIMIT 10;

2017-02-16-001-ORDER-BY-SQL-Statement-Example

 

Let’s try another example. What if we want to do an advanced / multiple sort? Let’s try it out, and change it up ever so slightly from the first sample query.

SELECT first_name,last_name

FROM customer

ORDER BY first_name ASC,

last_name DESC;

2017-02-16-002-ORDER-BY-SQL-Statement-Example-Multiple

If we scroll down to the first set of duplicate first names, we’ll see the last name has been presented in descending order (Z > A).

More Details About the ORDER BY Clause

Did you know? In PostgreSQL, you can ORDER BY columns that aren’t explicitly selected within the SELECT statement? E.g. We could only select first and last name, but order by their address_id value. Interesting, yes? Important caveat, other SQL programs, such as MySQL, may not let you do this.

Below:

SELECT first_name FROM customer

ORDER BY last_name ASC;

2017-02-16-003-ORDER-BY-SQL-Statement-Example-Other-Sort

Wrap-Up

Thanks for joining me and hopefully you’ve learned the essentials for the ORDER BY clause. Feel free to check out the entire journey on how to learn SQL, thanks!