Decoding the Vintage Shetland Project

I've been waiting a long time to be able to share this guest post from my husband and creative partner, Gavin. When being interviewed about my books I frequently say that there would be no books if it wasn't for Gavin and his 'behind the scenes' role in their creation. And this has never been more true than with the Vintage Shetland Project. The project has taken over four years, without Gavin it would probably have taken another five!

For those of you who don't know Gavin, he is a graphic artist and designer who worked for over 20 years in the print and design industry before I kidnapped him and lured him into the knitting world to be my partner, continuing to use his graphic design skills to create our self-published books. We both split our lives between the business and the farm and have often struggled financially, however Gavin has never suggested we stop doing what we do, he has never tried to stop me pursuing my dreams, my desire to create better and better books and my endless quest for more research opportunities. He has never once said four years research was too long, too expensive, or self-indulgent. He is endlessly supportive, creative, inventive and tireless and without him I could not do what I do. 

So over to Gavin who has written a fabulous piece explaining his significant role in the Vintage Shetland Project. 

"When Susan sets out on the process of starting a new book it isn't long before it turns into a joint effort, with myself coming on board for the graphic art side of the project. My role is the creation and layout of the books - preparing the pattern text for styling and typography, editing the photos for colour balance, applying the colour correction needed, preparing any diagrams, schematics and charts, and lastly when everything is complete and ready to go, I create the press-ready CMYK files that will be sent to the printers.

At the early stages of the Vintage Shetland Project was the process of 'reading the garments'. This involved Susan closely studying the garment so as to record the construction and patterning stitch for stitch. We needed a way to transcribe the colour patterns of the many motifs that make up a single Fair Isle sweater. A daunting task in itself, but magnified hugely when the idea of creating a book full of these garments was decided upon.

After some discussion we came to the conclusion that the most productive method would be for Susan to analyse each stitch and dictate it's colour out loud to me, where upon I would sit at the laptop to record the result.

At this point it was obvious to me that I was no longer getting the holidays in Shetland that I'd envisaged - the fishing expeditions and the sightseeing trips I'd planned while Susan was to be busy working at the Shetland Museum Archive - now, I too would be 'busy' with the project.

We needed to devise a clear process, so we would be able to understand and make sense of all the transcribed notes we'd taken when we came to look back on them, at a later time. This was something that I felt needed some careful thought and I had some ideas that would help in the long term.

Along with my many years of graphic arts experience I've also enjoyed computer programming at a hobby level, and have found myself writing many programs to aid me in my work role, often to process photos or create images. In fact, whenever I find myself doing repetitive task at the computer I look to see if I can write a program to automate the process. It has to be said that the process of writing a program can often take more time than it would have, to have just sat there and completed the monotonous tasks that needed doing, but it's the challenge, the problem solving and the creation process that is the enjoyment of the programming - along with that 'Thank Goodness' feeling when it actually works.

Creating a Code

There's a whole host a ways to approach the process of copying the Fair Isle motifs in a garment. I'm sure most people will be familiar with the idea of sitting with a sheet of 'squared paper' or graph paper and a bunch of coloured pens and setting to colouring in the squares - indeed that is a process many Shetlander's have done for years and we've seen many personal collections of motif stitch patterns. Up until very recently, knitting was still taught in schools on Shetland, where children would drawn up motif charts on paper. But we needed a method that would be quicker than this. A method that would create charts that we could reproduce on a computer suitable for printing within a desktop publishing package. So a graphics package would seem like an obvious choice then, or even a knitting stitch design program - but I felt these options would be too slow and restrictive for the amount of work we had to get through. I decided upon a simple code that I could type out as we worked though each garment. That's right, the humble text editor was my choice of software for this task and a simple but well defined code that I could deal with at a later stage, and write come computer code to read it back.

Looking at a selection of garments from the Museum's collection I started counting the number of different colours of yarn each one used; 4, 5, 8 and one with 10 colours. Perfect I thought (being a nerdy programmer type) 'Hexadecimal'! That would give us up to 15 different colours in one garment. For those of you that don't know (and actually want to know) hexadecimal is the name in mathematics given to base 16, also often referred to just as hex. Unlike decimal (base 10) that is represented with the characters 0 through to 9 for the units, the hexadecimal units are represented by 0 through to 9 followed by A, B, C, D, E and F to represent numbers 0 to 15.

I'm sure there are some of you at this point asking 'but why hexadecimal?' (if there is, at all, anyone left reading this!) - all will be explained very soon.

The idea was to give each colour in the garment a number then type that number into the text editor to represent the stitch in the motif. As the motif was transcribed by Susan I would end up with a textual representation of it - something along the line of this:

As you can see, one character represents one stitch. So the reason for using hexadecimal as our code meant that if we get a colour 12 it is represented with the character C, which is still a single character wide. If it was represented as the decimal number 12 it would be two characters wide (a one and a two). As well as it messing up my nice evenly arrange textual grids on screen, a '12' would be difficult later for the computer to read back. How would it know we meant colour twelve and not colour one followed by colour two. Also, computer systems and many programming languages are used to dealing with hexadecimal numbers (as it maps to binary in a more convenient way than decimal does) making it easier for the programmer to use hex than one might first expect.

So off we set, on the process of reading and transcribing garments. First, Susan would write some detailed notes in her note pad about the construction, together with taking lots of measurements and jotting down as much information as possible together with sketches that would enable it to be reconstructed and made into a pattern. At the same time I would take stacks of photos. First a few images of the whole thing front then back. Then more detailed photos of the finer points such as the welt, cuffs, sleeves, collar, sleeve heads etc. Finally finishing up with a close up of each and every motif in order from the bottom up, so we had something to refer to later when we created the charts, once we were no longer in Shetland and no longer had the real thing in front of us to check against.

Next came the coding process: I'd open up a new file in the text editor and make a few notes of my own, such as the garments code that the museum has assigned to it for their database records, together with a brief description and some of the same notes as Susan had made, so we could cross reference our results later. Then, studying each yarn colour we'd give it a code number and compare it with modern day shade cards to see if there was a match to currently available yarns, both in colour and weight - which would all get typed in.

Starting with the first motif Susan would read out the number of the colour, stitch by stitch, beginning at the bottom right hand stitch working to the left, then up to the next row once the first row of the motif had been dictated. Meanwhile I typed the numbers into the text editor, but mine were reading from left to right, and top to bottom, which could be confusing at times when comparing our results.

As many motifs were horizontally symmetrical, i.e. after the centre row the pattern would mirror itself in reverse order, we developed a system that Susan would tell me when we were on the centre row and I would type a TAB followed by a c to denote that this was the centre row. Which saved a certain amount of time.

On and on we went, a motif at a time until the entire garment had been notated. Often taking two to three hours to complete one piece. Some days, we could spend the whole day in the museum archive and only get through two items.

Decoding the Results

We now had a code, but a code is not much use unless it can be decoded! So I set about analysing the data we'd obtained to create a computer program to do something with it. Namely, create a coloured chart ready for the knitting process.

Some say an important decision at the early phase of software development, even before the conceptual process of 'what will it do' and 'how will it do it', or the feasibility of 'can it be a marketable commercial product', is to come up with a really cool, snappy name - or at the very least a really good project code name. After all, would Photoshop be that well known if it was called 'The Painty Program'? Well, as it turned out I couldn't come up with that snappy, cool name, so Fair Isle Decoder is what it became.

Anyway, back to the actual programming task: The first stage was to discover the colour codes we'd assigned. The program looks through the text file until it finds the colour references and the description of the colour, which it display in a window in order. But a very basic colour description such as light blue, isn't much use, so alongside each name is a coloured icon that allows the exact colour to be set for that yarn. When all the desired colours have been set the colour definitions can be stored so that they are remembered for future use, or edited at any time.

The final task is to work through the file and draw the chart for each motif that's been transcribed. It finds the start of the motif which in the file is denoted with the text #start followed by the description or title of the motif that we've given it - usually a simple M1 for the first motif or Peerie etc. It reads each line in order until it finds the text '#end'. Each line is then analysed character by character (stitch by stitch) and each colour number is converted into the appropriate coloured square in the chart, remembering to reverse the order from left to right - top to bottom into the knitting chart of right to left - bottom to top. If it discovers a 'TAB c' at the end of a line then it has to treat this as the centre row and reproduce each of the previous rows in reverse order.

There was one last thing to add to the program and that is the ability to create a key for the charts. This draws the colour square with the name of the yarn colour along side.

It took a fair amount programming time to get to this stage, but there we have it, each motif is produced as a colour chart, in a vector graphics format, that can be further edited in a standard graphics package. Job done!

... well not quite!

The next stage in the evolutionary process of taking the information we had gathered thus far and turning it into a modern multi-sized knitting pattern now starts to get complicated. That's right, all that previous work turned out to be the easy bit. Yes, very time consuming, long winded, painstakingly laborious, hard on the eyes and extremely tiring by the end of each day, but nevertheless a lot less problematic than what was to follow, and I have to admit, that on other projects and books Susan has done I've never been too much involved with this stage. I am of course referring to process of multi-sizing. During this process a number of the garments brought about some interesting and difficult mathematical problems, inherent because they were never intended to be re-sized. They were personal creations, one offs, even possibly, somebody's own knitting experiment. To investigate how to solve some of these mathematical conundrums we decided we would test things by creating the piece in a virtual way.

I took the charts that had been created by my program and dropped them into a graphics package, there I repeated them across the page and arranged each motif and peerie in their correct order to create a virtual fabric of stitches that we could manipulate, cutting away and adding stitches where needed, inserting and removing whole rows until we achieved the sizes required and still keep the integrity of the original garment.

This process worked very well, but it was a lengthy procedure and took quite some time to move sections around just to insert the odd row here and there. It prompted me to think about how the process could be done differently, in a way that could easily be changed and edited but reproduced easily if, for example, we decided to change one of the colours in the charts. The process of creating these full page graphics of all the motifs together, with me at the controls of the graphics package and Susan working from her mathematical calculations, telling me how many motifs across, the number of rows to add of plain colour between between set of motifs made me realise that what was actually needed was a textual 'description' of how it should look, that could be easily edited and then adapted for testing other sizes.

My Fair Isle Decoder program already had most of the code I needed. It could produce individual charts and a key, so it just needed some extra method of being told how to draw everything required all on one page. Trying to describe how a page full of charts are positioned could possibly be a very lengthy and wordy process, but to me (being the nerdy computer type that I am), an obvious way to approach it is to handle it like it's a computer program itself. Or more accurately a scripting language. And so the 'Visualiser Script' was added to my Fair Isle Decoder.

Visualiser Script

The basis of the scripts are themselves just simple text files, containing descriptive commands, or functions to instruct it to draw what we want, where we want it. As a very simple example, lets say, we want the first motif repeated 10 times across the page, followed by the Peerie motif then the second motif. The motifs are numbered sequentially in the order they are written in the initial file we originally transcribed. So if the peerie was the 8th one described we would create a script as follows:

motif(1,10)

motif(8,10)

motif(2,10)

A whole host of functions allow some very complex design to be quickly and easily constructed. I'm constantly adding more functions when I come across a need for it to do something extra.

Some of the more commonly used functions are:

motif() - draws a motif, with the option of it being repeated many times.

partmotif() - draws just a part of a motif, with full control over which part is drawn.

plain() - draws a block of a single plain yarn colour for a given number of stitches and rows.

There are plenty of others that are pretty much self explanatory from their names, such as:

repeatbox(), text(), line(), position(), arrow(), key().

The emphasis behind it all, is that it can produce a graphic image that is ready to be included in the book, at a quality that's ready for printing, and that no further editing needs to be done in a graphics package before it can be used.

To date, the entire program stands at a total of 14,583 lines of computer code, an untold number of hours and still no fishing!"

I'm sure you'll agree with me that Gavin's programme is in itself a truly amazing development and without it the progress on the Vintage Shetland Project would have been infinitely slower and Gavin really deserves a huge, resounding cheer from us all!

If you would like to support the Vintage Shetland Project and pre-order your copy of the book please go to my pubslush page. The campaign runs for another 2 weeks and is the only way to be sure of getting a copy of the book before Christmas.

for now

Susan xx