‘Found Poem’ Generator – Loading Data From A File

#poem #poetry #python #programming

The next stage in my Found Poem Generator project was to work on item number 6 in my list of enhancements. This is to update the program to read in the found poem lines from a csv text file rather than having them hard-coded in the program itself:

1. Output the created poems to a larger screen ie a graphical type large fonted output rather than  using the console window.
2. Provide two methods of running:
a. Auto-generate a new poem every 5 minutes or so
b. Provide buttons for user input to create poems on demand
3. Package the program up to be self-contained
4. Deploy the program onto some sort of platform that wasn’t my Mac laptop ie a Raspberry PI and large screen monitor or TV
5. Make the program auto-run when the device is started / restarted

Further enhancements could then be:

6. Use a text / csv file to load in the words and lines of the poems rather than the hardcoded ones used currently
7. Check the first letter of each line to capitalise it for poems and un-capitalise it for haikus, and add commas to the end of lines if they are missing or remove for haikus.

This tutorial got me most of the way there, opening the csv file and reading in the records. I could then access the individual fields of each row using the ‘row’ command ie ‘row[0]’ gives me the data from column 0 of the current row, ‘row[1]’ gives me the data from column 1 etc. I was then ready to read the lines from the csv file into my data structure in the program. This is where I came a little unstuck.

 

My project was based on each poem line or phrase being an individual object. Each object was created one after the other and given a sequential name using code like this:

p1 = phrase(blah, blah, blah)
p2 = phrase(blah, blah, blah)

So ‘p1’ becomes an object called ‘p1’ which has some attributes ie the poem phrase or line, the word and syllable counts. Once all the objects are created they get loaded into a registry so I can keep track of them and mark them off as used once they have been selected and displayed for a particular poem. My problem here was auto-generating the ‘p1 =‘ part of the code for each record in the csv file. 

There just didn’t seem to be an easy way of generating the new object names in order to instantiate objects for them. A re-think was required.

In my previous life I would have read the data from the csv into a multi-dimensional array and worked with that, so I thought I would explore this option first. Unfortunately Python doesn’t appear to support multi-dimensional arrays. It has lists which are single-dimensional arrays and dictionaries which are more structured versions of lists but in order to make either of these multi-dimensional one has to make a list of lists or a dictionary of dictionaries which started to look quite complex for my dataset which has 4 columns plus the ‘line has been used’ flag in the registry. So I googled for possible solutions and came across someone asking a similar question and one of the answers intrigued and surprised me – Python has SQLLite built in and it can be instantiated and run solely in memory. 

Neat. Now all I have to do is chuck out most of my code and rewrite using a SQL table in memory instead of instantiating all those phrase objects and using a registry to track them all.

This is of course a lesson in how not to develop, or one of the pitfalls of having bespoke software written, and potentially a pitfall of the agile software development methodology. If we don’t know at the start where we are going to end up then we can end up writing ourselves into a corner. Ever asked for what you feel like would be a small simple change to a piece of software and been quoted an extortionate amount of effort for it? This is where I was right at this moment, my whole design no longer supported the new functionality I need to write into it.

In reading the poem phrases in from a file I could no longer use my object based data structure. Now that I was no longer using this structure it meant that my method and code for selecting the lines of poem no longer worked and were no longer appropriate either. 

I began working methodically, finishing the function that reads the data in from the csv file and then working through the two functions that created the poem output and the Haiku output. Once I had finished the poem function, I was able to reuse much of the code to update the haiku function.

The previous methods used a random number generator to pick a line number and then simply iterated through all the poem line objects until it reached that line and then checked a flag to see if it had already been used. If it hadn’t been used before the line was selected and the flag updated. Either way the process then ran again until all the required lines had been selected. This sort of continuous scrolling though data is just about passable as a solution when working with a small array of data in memory, but the conversion of the data storage to a SQL based system meant this method was not really best practice. Sure it would work but it wouldn’t be pretty under the bonnet. It just would not do.

The change was relatively straight-forward, the random line number is still generated, the line is read directly from the table using a SELECT and then checked for validity. If it passes it gets used, if it fails then we go round again until the requisite number of lines have been retrieved. After working on this method for the ‘poem’ function I made it a little more efficient by updating the SQL to only select lines that had not been used and were the correct number of syllables. This still meant the program spent an amount of time firing ‘SELECT’ statements and returning empty recordsets when they didn’t match the right criteria ie the random line selected had already been used or didn’t have the required number of syllables. An update for the future on this method would be to firstly identify all the valid lines and then select one randomly from that subset, this would bring the SQL interaction down to a maximum of two SELECTs per poem line rather than a variable amount which is happening now.

This version also contains a little tidying up, I’ve learnt more about the ‘Pack’ command and organised the screen a little better and also change the font size based on the number of lines of poem that need to be displayed – this prevents the buttons disappearing off the end of the screen on long poems. In the back of my mind I’m also wondering what will happen if I have long lines as well, these are likely to disappear of the the sides of the screen so I ought to write in some sort of dynamic font sizing routine to check for and handle these potential issues.

The updated source code has been placed in Dropbox for those of you enjoying working through it yourselves.

One final thought I had to add to the ‘To Do’ list is for the program to output each of the poems created to a text or log file to create a permanent record of all the combinations created.

 

 

 

Found Poem Generator – User Interaction – Adding Buttons

Working through my list of enhancements for my ‘Found Poem’ Generator that I documented in a previous blog post: 

1. Output the created poems to a larger screen ie a graphical type large fonted output rather than  using the console window.
2. Provide two methods of running:
a. Auto-generate a new poem every 5 minutes or so
b. Provide buttons for user input to create poems on demand
3. Package the program up to be self-contained
4. Deploy the program onto some sort of platform that wasn’t my Mac laptop ie a Raspberry PI and large screen monitor or TV
5. Make the program auto-run when the device is started / restarted

Further enhancements could then be:

6. Use a text / csv file to load in the words and lines of the poems rather than the hardcoded ones used currently
7. Check the first letter of each line to capitalise it for poems and un-capitalise it for haikus, and add commas to the end of lines if they are missing or remove for haikus.

This blog post describes the update for item number 2(b):

2. Provide two methods of running:
a. Auto-generate a new poem every 5 minutes or so
b. Provide buttons for user input to create poems on demand

My previous experience with programming GUIs started a long time ago with Microsoft Access 2 and Visual Basic 4 and then, later, a number of Borland Delphi iterations with Microsoft SQL Server. With these development environments you got, well, development environments and that meant a WYSIWIG screen designer and a palette of buttons, text boxes, counters and other widgets to choose from. Not so with Python.

Tkinter Screen Layout Design

I had a google around for a WYSWIG screen designer but there really doesn’t seem to be anything that is recommended, one suggestion was to use graph paper but I’m not sure this project is complex enough to go that far. It seems that hand-coding buttons and labels is the way to go using one of the three Layout or Geometry Managers that Tkinter possess. These are Pack, Grid and Place:

‘Pack’, I’m informed, is the easiest to use but gives least control over placement of widgets, merely their relative positions to each other.

‘Place’ is used explicitly to set the position and size of a window, either in absolute terms or relative to another window. This didn’t sound like the sort of thing for creating and laying out a bunch of buttons.

‘Grid’ is similar to Pack but allows a certain amount of control of the placement of widgets by allocating them x and y co-ordinates within a notional two dimensional grid. The dimensions are then determined by the Grid Manager. This sounded the most likely candidate to try and use.

Tkinter Command Buttons

So now I have an inkling of how to place my buttons, my next task is to find out how to create the buttons and then assign the ‘poem’ and ‘haiku’ commands to them such that I could generate some interactive output.

Turning to the code, the first thing I found was that the label I was using to place the text on the screen, using an example copied from the internet, was using the ‘pack’ command. In order to keep things simple, I decided to change course and stick with the ‘pack’ function and see what the outcome was before deciding if I needed to get more complex with my button placement.

In the end I created three buttons and a slider bar, the buttons were for quitting the program (just to make things simpler for testing, I’ll remove this one later), a ‘haiku’ button and a ‘poem’ button. The slider allows the user to slide the scale from 1 to the number of ‘found’ lines there are available before pressing the ‘poem’ button to generate a poem of that number of lines. 

FoundPoemGenerator_Screen_Buttons

In doing this I uncovered what I thought was a bug in the system where after generating one haiku the program would appear to freeze with the spinning pointer icon, yet the ‘poem’ button worked fine. I say it worked fine, a little more button pressing revealed that the ‘poem’ button also eventually ran into the same problem, and then the penny dropped. The code is designed to mark any lines used in a poem as such ie already used, this meant that the ‘haiku’ function was very quickly running out of 3 and 5 syllable lines to use and so the program got stuck in an endless loop of looking for lines that weren’t there.

The fix was easy, to clear the ‘line already used’ flag between each press of a button. I’m not altogether happy with this outcome as my design was to ensure each poem produced was unique, which logically thinking couldn’t actually work unless I had infinite lines of poem to draw from. A case of having to admit my design was flawed and forge ahead with the new working design.

Source code available here for this update.

Found Poem Generator – Outputting Text To A Graphical Screen

Following on from my last post regarding the proposed additions to the ‘Found Poem’ Generator I started working on the list of enhancements with the aim of building the Found Poem Generator into something that could be used ‘out in the wild’ as a stand-alone art installation:

  1. Output the created poems to a larger screen ie a graphical type large fonted output rather than  using the console window.
  2. Provide two methods of running:
    1. Auto-generate a new poem every 5 minutes or so
    2. Provide buttons for user input to create poems on demand
  3. Package the program up to be self-contained 
  4. Deploy the program onto some sort of platform that wasn’t my Mac laptop ie a Raspberry PI and large screen monitor or TV
  5. Make the program auto-run when the device is started / restarted

Further enhancements could then be:

  1. Use a text / csv file to load in the words and lines of the poems rather than the hardcoded ones used currently
  2. Check the first letter of each line to capitalise it for poems and un-capitalise it for haikus, and add commas to the end of lines if they are missing or remove for haikus.

 

This post deals with number 1:

  1. Output the created poems to a larger screen ie a graphical type large fonted output rather than  using the console window.

The first task here was to identify a way of actually doing this, Python is a modular language where a lot of functions exist in external libraries that need to be added in to what ever you are writing. A search on the internet suggested Pygame or Tkinter might be the way to go. Pygame is a game development library whereas Tkinter is a GUI development library. Although neither jumps out as being ideal for simply outputting text to a large screen, Tkinter had the benefit of being built-in / pre-installed with Python so that was my first choice.

Another search on the internet, now looking for how to output text to a graphical screen using Tkinter, turned up a code snippet that I used to very quickly update the code to output to the graphical screen:

1-OutputToLargeScreen

As a proof of concept, this is working great for me at the moment, although there does seem to be some kind of bug or issue where the graphical screen doesn’t display or close properly. This appears to be a Mac specific issue, so I’m not too concerned at the moment as the project will ultimately be running on a Raspberry Pi, although I can foresee some more possible challenges with developing on a Mac and then porting to the Pi.

So onto the next stage now, providing buttons on the screen to allow prospective users to interact with the program and generate their own poems.

 

Source code for this update to the project can be downloaded here.

Found Poem Generator – The Next Level

When the latest Ledbury Poetry Festival e-mail dropped into my inbox last week it set me thinking again about the ‘Found Poem’ Generator I had been working on using Python. The festival was announcing the debut of their collaboration between poet Sara Jane Arbury, Joined Up Heritage and Ledbury Poetry Festival who have created a poetic walk around eight key heritage sites in Ledbury using ‘found poetry’ comprising words from First World War poets and phrases contained in the buildings themselves.

Ledbury has quite a history with poetry, being the birthplace of Poet Laureate John Masefield and has close associations with the Dymock Poets.

With my current interest in ‘found poetry’, following the course I participated in on FutureLearn featuring Michael Simmons Roberts and Helen Mort, and my latent interest in Ledbury and the Ledbury Poetry Festival having lived in Ledbury for a short while, this set the thoughts rolling around in my head: what if I could extend my Found Poem Generator to create a display piece or interactive kiosk? Could this then be utilised as a poetry installation out in the real world?

So, in order to achieve this I need to:

  1. Output the created poems to a larger screen ie a graphical type large fonted output rather than  using the console window.
  2. Provide two methods of running:
    1. Auto-generate a new poem every 5 minutes or so
    2. Provide buttons for user input to create poems on demand
  3. Package the program up to be self-contained 
  4. Deploy the program onto some sort of platform that wasn’t my Mac laptop ie a Raspberry PI and large screen monitor or TV
  5. Make the program auto-run when the device is started / restarted

Further enhancements could then be:

  1. Use a text / csv file to load in the words and lines of the poems rather than the hardcoded ones used currently
  2. Check the first letter of each line to capitalise it for poems and un-capitalise it for haikus, and add commas to the end of lines if they are missing or remove for haikus.

Subscribe to my blog and watch out for these enhancements and future posts – coming soon!

 

Found Poem Generator – Adding Haikus

Haikus

The wind blows in gusts
little pieces here and there
the fools in the hall

– Shadow The Poet aka James Laurie

Now that I had the Found Poem Generator working and creating poems with the number of lines asked for by the user I thought it was time to move on to the Haiku Generator idea. This would be a similar process but uses the syllable count property of each line of text. I also have the word count stored as a property which I may use later to create poems with lines of equal word length.

The Haiku has a fixed format of three lines, the first is five syllables, the second line is seven syllables, then back to five syllables for the last line.

I already had the code that generated a poem by lines, so I thought I could re-use this piece of code with some small modifications. This is always my methodology when coding – can I re-use something I’ve already done? It’s much easier to copy and paste something used in a previous project or something written in the current project rather than having to write a new section from scratch. 

Functions

When re-using code from the current project my second question is – can I modify the existing code to re-use it ‘in-situ’ rather than copying and pasting it out to the new location. What I mean by this is – can I make it a generic function? For example, we might have a piece of code that adds 1 and 1 together:

result = 1 + 1

If I needed to added two numbers together elsewhere in my program, ie 2 and 2, I could copy and paste the code above and update it to the new requirement:

result = 2 + 2

Nice and simple and works just fine, but, for me, the most elegant and de-buggable code always avoids duplication. Let’s suppose this was a more complex calculation, maybe 10 or 20 or 100 lines long, and I had used it hundreds of times in the program. Then imagine I found a bug in my calculation. Now I have to trawl though dozens or hundreds of lines of code to find and correct the problem wherever I found it. The risk being that I miss some and have now created for myself one of those annoying bugs that only appears under seemingly random circumstances and becomes harder and harder to track down. So, in circumstances like this I would always try to create a function from the original code rather than copy and paste it verbatim:

define adding_together_function(input:number1, number2, output:result):
result = number1 + number 2

Now this new function can be used whenever two numbers need adding together –

the_answer = adding_together_function (1,1)

Creating The Haiku Code

So with this in mind, I firstly updated the program to take the ‘doing’ code for the poem command out of the main loop of the program and put it into its own function. Once this was done I took a copy and changed the code so that it would select lines based on syllable count rather than just maintain a line count:

def output_by_syllable(no_of_syllables):
    found_line = 0
    while found_line == 0:
        random_num = random.randint(1,no_of_lines)
        phrase_meta = ("p" +str(random_num) )
        for phrase in Phrase._registry:
            if str(phrase.name) == str(phrase_meta):
                if phrase.get_used() == False:
                    if phrase.get_syllablecount() == no_of_syllables:
                        phrase.set_used(True)
                        found_line = 1
                        print(phrase.text)    

With an eye to adding other syllabic poem forms to the program in the future, I then created a Haiku function that uses the ‘output_by_syllable’ function to create the Haiku:

def output_haiku():
#line 1 - 5 syllables
    output_by_syllable(5)
#line 2 - 7 syllables
    output_by_syllable(7)
#line 3 - 5 syllables
    output_by_syllable(5)

Finally I added a new menu option for generating the Haiku:

    elif command == "haiku":
        output_haiku()    

I’ve added the updated code to my Dropbox share for this new feature.

the fools in the hall
little pieces here and there
The wind blows in gusts

– Shadow The Poet aka James Laurie

 

See a run through of progress so far:

 

Thinking On

Every day I walk these streets I try to take them in. Every detail, the pall of smog from the incinerator smokestack separating the oily black of the night sky from the neon glow of the streetlights lining the A50 below. The football stadium stands like a citadel in the middle of the middle distance and the drone of the traffic rises up to meet me.

Chamberlain Avenue draws up from London road in a exponential steepness dragging its way upwards from the kebab shops and oatcake shops, broken windows and broken paving slabs. The discarded sweet wrappers, beer cans and broken glass are suddenly gone and fallen leaves take their place in piles of brown and gold. Council workers blow and sweep the leaves into the back of their pickup but they don’t pick up the rubbish in London Road. Chamberlain Avenue leads to Penkhull, London Road is Stoke. Stoke Town. One of the six towns of Stoke-on-Trent.

I take in these details, because one day when I have left here and find my peace, these things, these memories and sights and sounds will become words and stories and songs. But for now they weigh heavily and it is the best I can do to take them in at all.

Found Poem Generator – Introduction

“How do you go back to the beginning?
Short stories are the best,
It’s always one step behind me,
The fools in the hall.
When your life has already flown?
Our life, a wheel within a wheel.”

–  Shadow The Poet aka James Laurie

I recently undertook a course on learning Object Oriented Programming using Python. This is one of the many useful little courses provided by universities and hosted by the FutureLearn website which I occasionally frequent. I wrote about their Song Writing course featuring Martin Simpson a little while ago.

I spent a period of my life writing software using Borland Delphi, so I am familiar with Object Oriented Programming, or OOP as it is known using its TLA. My interest in this course was to get stuck into a language that is relevant to the hobbyist / creative and as pervasive as BASIC used to be back in the good old days of home computing. Python is built in to Linux, so features on many Raspberry Pi’s, and also is therefore MacOS and so fits the bill straight off. What also piqued my interest was that the course would teach using Python by using it to write a text-based adventure game, something I misspent an early part of my youth playing.

As I followed the course I had an idea of using what I had learnt so far to write a ‘Found Poem Generator’. Rather than creating dungeon rooms and items as Objects, I would create lines of a poem. These lines could then have attributes attached to them such as word count and syllable count, perhaps even metre or some other attribute. Once I had collected my lines, counted the words and syllables and fed them into the program I would then be able to generate poems. The results could then be edited and new poems would come into being. 

I felt there was precedent for this firstly as I sometimes do this anyway by looking at my book of fragments and pulling various lines out to create a new poem. Valerie Laws did something similar in 2002, although more organic than technological, by writing her lines on sheep and using the random wanderings and grouping of the sheep to create new poems.

Warm drift, graze gentle, White below the sky, Soft sheep, mirrors, Snow clouds.

– Valerie Law’s Sheep

FoundPoemScreenShot1

At the moment the program can create a random poem based on my found lines which stored in one of its files. The command ‘poem’ followed by a number will cause it to generate a poem of that many random lines. I also store the number of words in the line and the number of syllables too as my next update will be to start incorporating standard poem forms into it. Haiku’s and other syllabic forms will probably be first.

The found lines are currently hardcoded into the program, as I still haven’t found out how to read data from a text file or spreadsheet yet using Python. In order to simplify things for myself I created a spreadsheet that I can use to hold my found lines. It has some text manipulation formulas in it that convert the lines I enter into the correct format for copying and pasting into the program file.

I have uploaded all this into a Dropbox folder here if you fancy a play with it, if you wouldn’t have a clue what to do with them then have a look at the FutureLearn course I did or look out for a future blog where I’ll try to explain.

“When your life has already flown?
Our life, a wheel within a wheel
The fools in the hall
The wind blows in gusts
Looking for that way home”

–  Shadow The Poet aka James Laurie

 

Pony Songs – Bo

‘Bo’ is a new song written this week, although the chorus was written a few weeks ago and had been lacking any decent verses to go around it. Even so, I’m not sure these verses go with this chorus either.

For a bit of fun I’ve run this through GarageBand to add a drum and percussion track on these two versions:

Strummed version:

Finger picked version:

The Street Where I Live – Part 2

It’s been an eventful week on Sheppard Street since my last post about the fly-tipping. I used the new Stoke-on-Trent City Council App to report the mounting pile of waste and I’m pleased to report the old flooring is gone, although the old mattress springs next to it still remain.

The pile of settee cushions, however, has been bolstered by the addition of a new mattress.

Before:

IMG_2835

After:

IMG_2862

More news from this intrepid reporter next week!

Update – all the mattresses and sofa cushions have been collected, plus one to the SoT Council and their app.

Chamberlain Avenue

Something is afoot at the top of Chamberlain Avenue, a rash of laminated notices appeared suddenly on all the lock-up garages.

IMG_2823.JPG

Eviction. Eviction to complement the planning application notice.

Chamberlain Avenue.jpg

Goodbye garages, hello houses. Not everyone received the news well though.

IMG_2860

So, is this good news or bad news?

Good news for the social housing shortage, bad news for the trees and grass.

IMG_2861.JPG

Good-bye trees.

Who puts their car in a lock-up garage these days anyway?