Tag Archives: python

Zato: Add MSSQL to Outgoing SQL Options

To Connect with MSSQL or SQL Server via Zato 3:

#sudo su - zato
$/opt/zato/current/bin/pip install pymssql
(More info: http://pymssql.org/en/stable/intro.html)
$nano /opt/zato/env/qs-1/server1/config/repo/sql.conf

Add the following lines at the end:
ping_query=SELECT 1

???I don’t know if this is required or not, but I still do it.
Deploy to other servers in the cluster:
$cp /opt/zato/env/qs-1/server1/config/repo/sql.conf /opt/zato/env/qs-1/server2/config/repo/

Restart your cluster and MSSQL should appear as an option under SQL Outgoing connections.

Zato MySQL Connection with UTF-8

Today this error appeared in an Zato Service:

UnicodeEncodeError: 'latin-1' codec can't encode character u'\u2019' in position 1: ordinal not in range(256)

I turns out that SQLAlchemy defaults to latin-1 encoding instead of UTF-8 for MySQL connections.

Thanks to Andrea on a Zato Thread here: https://mailman-mail5.webfaction.com/pipermail/zato-discuss/2015-November/001443.html points out that we can just add
dbname?charset=utf8 in the WebUI.

While not intuitive, it does work.
I added it, had the service try the action again and all was well in the world again.

SuiteCRM / SugarCRM Rest API 500 Error get_entry_list

I stumbled upon this problem today when trying out the SuiteCRM Rest API.
When searching for an account by it’s name I was using a search like this:

"name = 'Andrew'"

The API was returning a 500 error.
I dug into the suitecrm.log and found this:

where (name = 'Andrew') AND accounts.deleted=0: MySQL error 1052: Column 'name' in where clause is ambiguous

It seems that because of the joins, it can’t work out which ‘name’ we are talking about. Lets add the table name in and tada it works:

"accounts.name = 'Andrew'"

Zato and Pandas (strange combination I know)

I’ve been working with Zato for a project and I’m getting some random text out of a HTTP request that looks like a CSV file, so the easiest way I know how to deal with this CSV data is with Pandas, then I could process it enough and clean it up to hand back to Zato.

But Zato doesn’t ship with Pandas….
(TL;DR at the bottom of the post)

In theory this should work:

/opt/zato/2.0.7/bin/pip install pandas

But then I get the error when I do the following:

>>> import pandas as pd
File "/opt/zato/current/local/lib/python2.7/site-packages/pandas/__init__.py", line 19, in
"Missing required dependencies {0}".format(missing_dependencies))
ImportError: Missing required dependencies ['pytz', 'dateutil']

No luck.

After trying pip install pytz and pip install –upgrade pytz
I get this error:

Installing collected packages: pytz
Found existing installation: pytz 2014.4
Cannot remove entries from nonexistent file /opt/zato/2.0.7/eggs/easy-install.pth

But this does work:

/opt/zato/2.0.7/bin/easy_install --upgrade pytz

Now to fix dateutil.

easy_install --upgrade python-dateutil

Let’s try it again:

>>> import pandas as pd
Traceback (most recent call last):
File "", line 1, in
File "/opt/zato/current/local/lib/python2.7/site-packages/pandas/__init__.py", line 23, in
from pandas.compat.numpy import *
File "/opt/zato/current/local/lib/python2.7/site-packages/pandas/compat/__init__.py", line 361, in
from dateutil import parser as _date_parser
File "/opt/zato/current/local/lib/python2.7/site-packages/python_dateutil-2.6.0-py2.7.egg/dateutil/parser.py", line 40, in
from six import text_type, binary_type, integer_types
ImportError: No module named six


easy_install six

And for a final test:

>>> import pandas as pd

Yay, it’s working. Hopefully, nothing in Zato has broken…


/opt/zato/2.0.7/bin/easy_install pandas
/opt/zato/2.0.7/bin/easy_install --upgrade pytz
/opt/zato/2.0.7/bin/easy_install --upgrade pytz
/opt/zato/2.0.7/bin/easy_install six

Selenium / Python Notes June 2016

For the past 24 hours I’ve been jumping into some Selenium fun in Python.


  • Firefox 47 doesn’t work, however Firefox 47.0.1 does.
Selenium Setup

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.proxy import *
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import TimeoutException

def init_driver():
driver = webdriver.Firefox()
driver.wait = WebDriverWait(driver, 5)
return driver

def lookup(driver):

if __name__ == “__main__”:
driver = init_driver()

Two ways to interact with an element on the page:

textbox = driver.wait.until(EC.presence_of_element_located((By.ID, "formTxtBox")))
except TimeoutException:
print("Login Form Not Found!")


textbox = driver.find_element_by_id("formTxtBox")

Taking HTML and feeding it into BeautifulSoup

div = driver.find_element_by_id("id_of_div_here")
soup = BeautifulSoup(propertyDropDown.get_attribute('innerHTML'), 'html.parser')
#Do something with the soup....

Creating a list of dates (requires pandas)

import pandas as pd
dates = []
index = pd.date_range('2015-7-1', periods=52, freq='W-WED')
for i in index:
return dates

Convert Table into CSV with BeauitfulSoup

(there are probably 100 better ways to do this…)

#Get the element of the table using CSS Selector
table = driver.wait.until(EC.presence_of_element_located((By.CSS_SELECTOR, "#table > tbody")))
#Take our table and pass it into BeautifulSoup so we can easily transverse it.
soup = BeautifulSoup(table.get_attribute('innerHTML'), 'html.parser')
#Make an empty list
rows = []
#Loop over the soup
for row in soup.find_all('tr'):
rows.append([val.text.strip('\n').encode('utf8').strip('\xc2\xa0') for val in row.find_all('td')])
#Save out to CSV.
with open('output_file.csv', 'ab') as f:
writer = csv.writer(f)
writer.writerows(row for row in rows if row)

Disclaimer: I’ve been programming in Python for about 5 minutes and using Selenium for about 2….while this code worked for my project, it’s most likely littered with errors and bad practices. This is mostly just a reference for myself for a project I might need to reefer back to this stuff in 6-12 months time. I am definitely not the person to ask for help from when it comes to this stuff.

Xero API pyXero and Anaconda Python

Personal TL;DR notes on getting this to work, not a tutorial.

Download Install Anaconda
– At the time of writing it didn’t create Windows Shortcuts, fix this with openning cmd as admin

‘c:\Anaconda2\scripts\conda.exe install menuinst’

Agree to Updates

‘c:\Anaconda2\scripts\conda.exe install -f console_shortcut ipython ipython-notebook ipython-qtconsole launcher spyder’

Agree to updates.

Now there should be shortcuts in:

C:\ProgramData\Microsoft\Windows\Start Menu\Programs\Anaconda2 (64-bit)

Now onto Xero

  • git clone https://github.com/freakboy3742/pyxero.git
  • Open up Anaconda Prompt (Anaconda2x64)
  • cd into folder with git of pyxero
  • use command: python setup.py install
  • pip uninstall oauthlib
  • pip install oauthlib==0.7.2

The last two commands fix the error: ImportError: No module named jwt.algorithms

Again these are personal notes, not a full blown tutorial.


Building CSS sprites using Python Glue

css-sprite was giving me some issues so I went looking for a different way to tackle the problem.

While a pain in the rear end to install, I found Glue. Given I already had all the python stuff installed it was pretty easy for me to get it working for this project.

  1. Install Glue http://glue.readthedocs.org/en/latest/installation.html
  2. Run Glue, here is the command that I like:
    glue src glue –scss –sprite-namespace= –namespace= –cachebuster

(The above command, looks for all the images in the src folder, and then outputs into the glue folder).

While the scss support is woeful, the retina side of things seems to be working better. (Although I’m not using the retina switch in the command above).