Using Geek Power for Good: Better Living Through Code Edition

bottle of Buffalo Trace bourbon

Buffalo Trace bourbon

Bourbon has become a hot commodity, and as it takes years to make, the supply can’t quickly ramp up to match demand. Buffalo Trace is one of many brands that have become quite popular, making it hard to find. The clerk at a local Virginia ABC store told my wife that they get a small shipment in and it “flows out like a river”, and is gone in a day. My wife found that you could check inventory of local stores online, and asked me to write a script to check it. Starting tomorrow morning, the “Buffalo Hunter” script will run once a day, check the inventory at our two closest stores, and send her a text if there are any bottles in stock.

Version 1 was a fun 1-day project. I had to learn some new tricks, as the page uses client-side javascript and I hadn’t used Twilio to send texts before, but I got it all working well. Some time later I decided to port it to the cloud, using the AWS Lamda service, which had a short but steep learning curve.

The website uses javascript to generate a dynamic page, so I couldn’t simply use something like Beautiful Soup to parse the html. So I used Selenium, using a Chrome headless browser on my local version, switching over to phantomjs on AWS. I switched to phantomjs because you need to have executables compiled to run under AWS, and I found a precompiled version of phantomjs on the web, and didn’t find the same for Chrome.

There was one other “gotcha” I ran into. I use Windows. While I had found a correctly compiled phantomjs executable, when I zipped it along with the other files to upload, it lost its permissions settings. I could have booted up in Linux, instead I installed the Linux subsystem that’s available for Windows 10 and used bash to zip the files up. That ended up working fine. You also need to change the directory for the phantomjs log to the /tmp/ folder that AWS gives you write access to.

In version 1, I handed off the final processed web page to Beautiful Soup because I hadn’t used Selenium’s parsing before, and I’d used Beautiful Soup’s. You can easily hand off the processed resulting web page from Selenium to Beautiful Soup (see the commented out line that starts page2Soup in the code below). When I moved to Amazon, I also figured out how to do the page scraping in Selenium, so that I didn’t need Beautiful Soup any more. The concept’s simple, but I didn’t find a good reference for the find_element)by_css_selector in python, so it took a little trial and error. .If you’re interested, here’s the version of the code that runs on AWS:

Buffalo Hunter.py


import logging
import datetime
import time
# from bs4 import BeautifulSoup
from selenium import webdriver
from twilio.rest import Client

accountSID='SID Here'
authToken = 'token here'
stores = {'219': 'Old Courthouse' , '231': 'Maple Ave.'}

# options = webdriver.ChromeOptions()
# options.add_argument('headless')
# driver = webdriver.Chrome('c:/program files (x86)/chromedriver.exe')
driver = webdriver.PhantomJS(executable_path="/var/task/phantomjs", service_log_path='/tmp/ghostdriver.log')

def myhandler(event, context):
	try:
		results = ''
		success = 0
		for store in stores:
			driver.get('https://www.abc.virginia.gov/stores/'+store)
			make_my_store = driver.find_element_by_id('make-this-my-store')
			make_my_store.click()
			time.sleep(5)
			driver.get('https://www.abc.virginia.gov/products/bourbon/buffalo-trace-bourbon#/product?productSize=0')
			time.sleep(5)
			element = driver.find_element_by_css_selector('td[data-title="Inventory"]')
			# page2Soup = BeautifulSoup(driver.page_source, 'lxml')
			# element = page2Soup.find("td", {"data-title": "Inventory"})
			inventory_value = element.text
			if inventory_value <> '0': success = 1
			results= results+stores[store] +' has '+inventory_value+ ' bottles of Buffalo Trace. '
		driver.close()
		driver.quit()
	# Send results if inventory not 0 at both stores
		if success == 1:
			results = 'Success! ' + results
			twilioCli = Client(accountSID, authToken)
			myTwilioNumber = 'myPhoneNumberHere'
			destinationCellNumber = 'destinationCellNumberHere'
			message = twilioCli.messages.create(body=results,from_=myTwilioNumber, to=destinationCellNumber)
	except Exception as e:
		logging.error(str(datetime.datetime.now())+' Error at %s', 'division', exc_info=e)

Necessity is the Mother of Invention

Not all solutions involve electronics or coding. I recently had rotator cuff surgery on my right shoulder and it’s more comfortable and better to sleep in a recliner for awhile after the surgery.  Unfortunately, like the vast majority of recliners, ours has its controls on the right side, posing a problem for someone in a sling.

Control buttons on my power recliner

So I put some PVC pipe together before the surgery:

PVC pipe “hook” to reach over with my left hand and work the control buttons

Close-up of the tip, where I glued on a piece of rubber to serve as a “finger.”

It’s a bit like lockpicking, you have to go totally by feel. While a bit awkward, it works!

Yorick 2.0: The Personality Split

Introduction

When Yorick was first brought to life, he had Alexa’s voice. A lot of his charm was the incongruity between his appearance and his voice. At the same time, a number of folks asked about having a creepier voice and I wanted to try to do that for this Halloween.  An update to the AlexaPi project added support for the SoX audio playback handler as an alternative to VLC. SoX has support for audio effects, so it became possible to change Yorick’s output voice. I didn’t want to lose Alexa’s voice, so I edited the AlexaPi code so that it would recognize both “Alexa” and “Yorick” as trigger words, with the output sound depending on which trigger word you used. As a result, Yorick now responds either as Alexa or with his own voice.

Just like Elliot on Mr. Robot, Yorick now has a split personality.

Conversations

I talked to Yorick, aka Alexa, a bit about Halloween:

It turns out that Yorick is a baseball fan and was rather disappointed that the Washington Nationals aren’t in the World Series. Awhile back, I asked him about going to one of the playoff games:

Technical Notes

AlexaPi uses PocketSphinx for recognizing the trigger word. The original code is set up to recognize a single trigger word or phrase, which you can easilly change in a yaml configuration file. However PocketSphinx can recognize multiple keywords or phrases selected from a python list. Some editing of the AlexaPi source code was needed in order to change the trigger from a single variable to a list. Similarly, the code was modified slightly so that once a trigger word was recognized it checks which word was used. If the trigger word is “Yorick” it changes the pitch and speed of the audio output.

I used version 1.5 of AlexaPi. This and previous versions had a problem in that the temporary file names used were the response code that the Alexa voice service returned. These sometimes included characters that were illegal for file names or that were too long for a file name. I patched these problems (and later versions of AlexaPi have fixed this problem).

In addition, the servo motion routines had to be modified slightly, as version 1.5 and later of AlexaPi begins streaming questions to the Alexa voice service as they are asked, rather than waiting until the question is finished. This results in a faster response time.