This portfolio documents the entire development process of the production piece for the assignment, and will go other the general process of creating the production piece, the research, the usage of software and its proficiency, as well as other aspects that may be critical to document during the creation process will be documented here within this blog.
This is the link to the video, containing both the narration as well as the final product via a screen recording due to the hardware limitations of my laptop not being able to properly render
Researching for the idea
Before the development process could actually begin, I began researching into what I was wanting to do in regards to the assignment, since a 360 video was what I was planning on making, so I had to choose something that delivered a unique experience to the user. I wanted to choose something that was relatively simple and easy to do, but had a deep meaning and could have an impactful experience on the user. It was admittedly a tight rope I had to walk, finding something easy but impactful, given that it would be my first experience.
Many idea’s were considered at first, such as a similar format to the cancelled Silent Hills demo P.T where the player would walk through a looping hallway, though this was deemed far too complex with my limited skill-set in a VR environment.
However, after some consideration, and a weeks worth of time dedicated to thinking about it, I decided to make my production piece based around the paintings of Ivan Seal, a British artist who is well known for making abstract art pieces based almost entirely around memory and creating imaginary objects.
Because of how his art is made, it is essentially next to impossible to directly tell as to what the object is that the painting is depicting in most cases, especially with how abstract that they are. An example of this would be his painting called “Allchav tart”, which shows the following.
What fascinates me about these paintings of Ivan’s is how much they essentially challenge the mind in trying to make sense of what is being presented to them, only for them to constantly second guess themselves when they have a rough idea of what the painting is supposed to be of. This is amplified according to an article on the guardian where Ivan was featured as artist of the week, stating the following:
“In fact, Seal paints largely from memory at the rate of a work a day, making his approach improvised and automatic. Yet with the paint applied in thick brushstrokes, or sculpted with a palette knife, the artworks themselves are very much of this world. They dart back and forth between reality and the distortion of things remembered.”
With these paintings almost being made entirely from memory, it does make each of Ivan’s paintings seem like their within their own reality due to how abstract they are in appearance, which adds to the mystery and uniqueness behind them.
One of the most well known examples of this, would be his artwork used for the cover of stage 5 in the album Everywhere At The End Of Time, the piece also goes by its own abstract name, being “eptitranxisticemestionscers descending”:
From what I’m looking at from this piece, I imagine it as like a woman in a large old fashioned dress, walking down a set of stairs with a cane, though given its abstract design in nature, it can be interpreted in any way the user see’s it.
It is because of the abstract and strange nature of Ivan’s paintings, that I decided to make it the basis for the VR assignment of this module, and come up with an experience where the user’s are free to interoperate what they are seeing, but aren’t able to come up with a clear and concise answer in figuring out properly what the object is.
Planning the scene
Given the unique subject that I was attempting to make (in an environment that essentially forces the user to guess at what their seeing before inevitably second guessing themselves), extensive planning had to be done in order to properly figure out how I was going to make this environment work at all.
I first started with a simple list of idea’s consisting of mostly what came to mind at first thought, and dropped them down onto the list. Whilst the list did not end up being as large as I hoped it would be, it would still have plenty of idea’s for me to process and decide between.
Out of these idea’s, I then decided to try and narrow it down in order to get to an environment that would be plausible to make for the assignment.
An abstract art museum seemed relatively easy at first glance, though the more I thought into it, the more I pondered on it being too simple for what I had in mind, so I went against that idea.
The imagination of the mind granted a lot more freedom, though it proved difficult to think of an exact starting location, since I couldn’t come up with a starting position or an environment which would fit. This would also affect the completely abstract world environment concept since that would have a lot of overlapping with the imagination of the mind, so both of those idea’s for environments were rejected.
A similar overlapping occurs with the final two potential environments, being a normal world that slowly becomes abstract, and something similar to the album “Everywhere At The End Of Time”. Both of these would be quite demanding, in having the environment change over time, likely having several different kinds of environments just bleeding into each other in order to make an immersive experience. Though the main concern with that kind of environment mainly comes from the latter idea, being ethical concerns if it was based on something similar to the album “Everywhere At The Time”. I mainly expressed these concerns, as the album in question, describes itself as:
“Everywhere at the end of time’ was a series exploring dementia, its advancement and its totality.
Audio remembered, disfigured and forgotten by The Caretaker.”
To summarise, the album would use samples of old ballroom music ranging from the 1910s to the 1930s, to create a representation of the advancement of Alzheimer’s disease/dementia, through the mixture of vinyl crackles, distortions, scrambling, etc, to create an impactful experience on the user. This album has known to be very powerful in terms of delivering it’s message, and from what I have observed, if a person is not within a solid state of mind, it may trigger an emotional response from the user, especially if said user has a relative that is a current victim of the disease, which becomes especially apparent within the later stages of the album. Whilst the album was made with the intention of spreading awareness for people to see what the disease could be like, it could also trigger emotional/stressful reactions out of the user that may cause them to become extremely uncomfortable, especially since in later stages, it uses multiple samples in such a fashion to give off the sensation of “brain fog” and immense confusion from the perspective of The Caretaker alias.
For these reasons, I decided that neither of the idea’s would be done, both for how advanced they would be in terms of demanding skill, and in general ethical concerns for the latter of the two, since it could end up triggering too much of an emotional reaction that causes immense stress and distraught for the user. This would leave one idea for an environment left to be used as a potential base for the assignment.
Whilst this last idea did share some overlap with the other idea’s that were rejected, I felt like having an environment based around the mind of an abstract artist, would not only fit with the vision that I am going for, but it would also allow me to conjure up a possible perspective, of how abstract artists such as Ivan Seal, come up with their pieces, as previously stated earlier, Seal’s paintings are created entirely from memory, with their own imperfections purposefully, which is what I plan on following when it comes to the creation of the environment, and as such, decided that this would be the perfect idea to follow along with for the assignment.
Planning the environment
Before going into maya to create the environment, I decided to conjure up a few rough designs for the general shape of the environment, the main area in which the user would sit in for the experience. I wanted to approach the style of the environment in a way, so that the user would immediately know where they are, but that would be all the information they would get.
I did have to take into consideration the distance that the abstract objects would have to be from the user in order to ensure that not only would they be at a safe distance for the user to not get disorientated, but close enough so that they would easily be able to follow the objects around them.
I then refined the rough designs of the environment to be more of what I was thinking of in mind, mainly in terms of the abstract objects that would be floating around the user, as they would only vaguely resemble that of day to day objects with a great number of imperfections as to deform them into abstract objects that users won’t be able to make out what it exactly it.
After some drafting and minor redesigns here and there, I was able to draft out a more fleshed out version of the environment that I am looking to create.
I was satisfied with these designs, not only because they were what I was looking for in terms of an environment, but they also in a sense, lined up properly with the functions of an actual brain. For example, the canvas in the environment, would be located at where the back of the brain is. This location was chosen in particular as this is where the occipital lobe is in a normal brain, which is what allows a person to observe and see the world using their eyes, hence why the canvas was placed at the back of the brain environment.
The same goes for where the user will be standing also, since the location of their standing position, is where the Thalamus is, which essentially the central part of the brain, in working the learning and memory processes of the brain (which also happens to work as where the user is in the environment due to its central location).
I also planned out a colour change for the environment for during the later parts of the video, where the abstract objects floating around would begin the clash into each other, which would create the following:
I thought of this special interaction as it would allow for me to make the video a lot more interactive and impactful rather than just having the abstract objects spinning around for a while before merging onto the canvas without having really any sort of change, so having not only the environment change colours, but also have a source of light, would not only have it be a more impactful experience, but also could symbolise the brain of an abstract artist piecing together the final art work that they are going to create and paint onto the canvas at the end of the video.
As for how the abstract objects themselves will be created, I plan on taking inspiration from many of Ivan Seals paintings, due to their abstract nature and design creating a sense of confusion, that makes the user constantly guess at what the object is that the painting is representing in an abstract like form. I was going to initially try the same method as how Ivan Seal came up with his paintings in making them entirely from memory, though I swiftly decided against it, since not only would it be difficult to try and remember an abstract object, but I’ve never done the process before myself, so it would require some time to learn, which with the limited time we have allocated for the assignment, would simply not be possible at all.
Instead, I tried taking inspiration directly from some of seals paintings into make some of my own abstract objects. I chose 3 of his paintings that intrigued me the most, and then tried to come up with my own designs for not only the abstract objects, but also the final abstract art piece that would be placed onto the canvas towards the end of the video.
Admittedly, this was the first time I had ever tried to draw any form of abstract art before, so my attempts were rather mediocre and underwhelming to say the least, though it was better to at least try and come up with some form of concept designs (even if they ended up looking like the first time I touched an art program in my life) so that I would at least have something to work with.
Planning object/camera movement
When it comes to making 360 video’s, especially when it comes to users who are actually viewing them with a headset instead of just in the browser, since you have to take factors into account such as motion sickness, cybersickness and so on, as to ensure that the user in question has a comfortable experience . If handled poorly, these could result in the user having headaches, nausea, light headedness and potential other symptoms that could negatively impact their experience.
I decided to ensure as to avoid as many of these issues as possible, camera movement would be kept to a minimum, and it would remain stationary throughout most of the experience, with the only form of movement being towards the end, where it would zoom in on the now completed abstract art piece.
These were intentional as to ensure that the user can have a comfortable experience and to avoid any form of motion sickness that may come to the user. Plus this allows the user to follow along to their hearts content, as well as look around the environment in a calm manner.
Blocking out the environment
With what I had in mind (both metaphorically and literally) now set in stone and drafted out in concept, I began the block out process that would go on to create the general environment. Blocking out is a critical process that can essentially shape out the entirety of an environment, as well as change how things are shaped out relatively easily due to the lack of any proper details.
Before beginning any of the initial process, I decided to create a trello board as so that then the monumental task of creating the whole thing into many sub tasks as to not only make the process much easier to manage, but to also document progress down the line, and to see how much I will be able to achieve with the project as a whole. Ahead of time, my goals would end up being a bit too ambitious with time constraints, but with the time I had, I attempted to get as many checks done on the trello board.
With the trello board now set, and the path laid ahead of me, it was time to begin the creation process for the environment that the 360 video would be taking place in.
I first started by mapping out the general size of the environment, as so that I would know how big of a scale the environment would be, this would be done by using 2 flat ovals to roughly represent the shape of the brain. I originally went with a much smaller size, though after placing in the sphere as a placeholder, realised that the size was far too small, and opted to significantly increase the size of the ovals to more match the environment that I was looking for.
With the size now confirmed and in place, I began the block out stages by first making the sphere into a low poly model, and then adjusting the nodes of the sphere into the rough general shape of the brain. This sphere would then be duplicated and then flipped as to save time on creating the general block out.
With the rough shape of the brain now cut out and ready to use as a guide, I made a duplicate of the brain, and cut it in half so that then I can get a rough idea of not only how much space would be inside the brain, but also so that I would know roughly where to place all the other objects within the environment, such as the thalamus, the canvas at the back, and eventually all of the objects. During this process, I had to make some minor adjustments to the general shape of the brain as to ensure that it would properly fit the objects that were going to be placed inside of it.
I would then make the block out for the thalamus, which would also server as the central platform that the user would stand on and look around in terms of the environment, since the camera for the most part will not be moving at all, up until the very end. The canvas at the back was also added along with a border around it that wasn’t in the original concept art, as I decided that it felt like it needed to be more directly connected to the brain, as opposed to just a few tendrils surrounding it in the concept art.
Then I assigned base colours to each of the objects within the scene as to get a rough idea of how everything would look, and this was the result that I got.
Story Boarding
With the block out now sorted out and done, I could move onto the next item on the trello board that I set myself to do for this week, that being the storyboard timeline of how the video will play out.
This step will be critical since it will be sued as a general timeline of events in how the video will play out, so I will have to make sure that not only it will make sense in how everything goes along, but also to take into consideration both how much there is moving within the environment as well, since if too much happens simultaneously for the user to keep track of, then it could end up up with an uncomfortable experience that may cause motion sickness to them, which is what I am wanting to avoid, since I myself also end up getting that sickly feeling when using a VR headset for more than approximately 10 minutes at a time.
I planned for the video to be approximately 3 minutes and 40 seconds in length with the story board going into detail as to how the video would play out, with notes on critical slides, as well as a key in terms of what each objected represented within the story board, as to make it easy for me to identify which objects represent what, as well as how the general flow and structure of the video will play out.
Obviously there will be a handful of changes down the line (as is the case with most story boards), since this is only a rough guide as to how the video will play out, and I do feel like I would make some subtle changes to the general flow of the video that are currently not in the story board. Though for now, this will serve as a general rough guide for how the video will play and be laid out.
With this part now also done, the trello board would be updated, and I felt like more could be achieved, so I decided to move one of the tasks I had set myself for next week over to this week, as to see if I could also get that one completed as well.
Creating the drone ambience
Creating the audio that would play throughout the entire project was something that I believed to be a crucial point in ensuring that it fit with the environment that I had. I preferably wanted to create the drone based audio on my own through the use of audacity, though my attempt turned out to be far from what I was hoping to make, though it did include some concepts that I did implement in future attempts. I will link the result of my first attempt below with the following instructions.
Whilst the end result wasn’t what I was hoping it to be, it was roughly on the right track of what I wanted in mind. It was at this point that I remembered about the album everywhere at the end of time, specifically, layer 2 of the last track, where it turns from a series of distorted sounds into an organ like drone. This drone organ, much like all of the other songs within the album (as well as the entirety of the caretakers discography), is sampled from an existing song and modified slightly in order to fit the nature of the album. The song used, is called “Präludium und Fuge in e-Moll / A Song of Sunshine” and is only slightly slowed down, and droned via reverb. This is how the track sounds in the caretakers album.
Giving the audio all of a single listen told me that it was almost exactly what I was going for in terms of a drone like ambience, and with it being mixed with an organ, made the audio sound a lot more angelic-like, which when paired with the theme of my 360 environment, made it a near perfect fit, I just had to do some minor modifications to the audio in order to make it fit into the environment.
I took the audio into audacity and began to play around with it, messing with the speed settings as well. I kept a note pad file open with all the setting adjustments made to the audio as a sort of path for me to follow, in the event that I would have to back track, which looked like this.
After giving the audio one final listen, I felt like I had managed to turn the audio into exactly what I was looking for, an angelic drone like ambience that plays in the background, as the player observes the environment moving all around them. This is the final result of the audio after all of the listed changes are applied to it.
With the main body audio now done for the assignment, I updated the trello board to mark it as complete for the set of tasks for the first week.
Making the assets
Now with the first week of tasks over and done with, it was time to get into the second week, which I decided to keep things relatively simple this time round with only 2 tasks. Being to create the assets for the scene such as the canvas n abstract objects, and to create audio in the event of a collision between 2 abstract objects, as well as for when the 3 merged objects at the end, merge onto the canvas at the end of the video.
Before getting to work on the objects, I felt like the size of the environment was not large enough, especially with how small movements inside the area whilst I worked on it would frequently place me back outside, that and I wanted to ensure that the abstract objects would have plenty of room to move and rotate around the camera for the user to look at, and then group them all together under the name “Block_Out” to keep things organised.
I also decided at this time to get the main body audio track imported into the scene, as so that I could use it as a rough reference point for how the video will play out, and how I can adjust it accordingly. However, importing the audio was proving to be a bit of a challenge, given that whenever I imported the audio into Maya, there would be zero playback at all. I tried to research and look up various different methods in order to troubleshoot the issue, though I was presented with no such luck, so I was left no choice but to leave the audio out until post production where it would then be added in afterwards.
I decided to keep up with the creation of the abstract objects, and just started to create the initial ones that would spawn into the environment. These would consist of various shapes that have been morphed into somewhat tangible objects similar to the likes of Ivan seal, as to give the user enough of a sense of familiarity for them to take a guess at what the object might be, but not enough as to keep them guessing at what the object might be.
Most would be made out of using a base object, pulling at the various vertexes around the object in various ways (either through directly moving them, to rotating them, recalling them, etc), and then applying a smooth effect to them just once as to make them not look as jagged and low poly. Then they would be duplicated and merged with other objects as to loosely create objects that exist in the real world just enough to make the user think they know what their looking at. A total of 6 of these abstract objects would be created in total, one of which would take direct inspiration from another one of Ivan’s paintings, being “beaten frowns after”
The base abstract objects would be kept relatively simple, since these would also be used in the creation process for the merged objects, which would involve 2 abstract objects being merged into each other, which would have to be done carefully since each of the 3 merged objects would make up the final piece at the end.
Each merged object would also have a few additional details added to each of them as to give the end result a more merged like result. For example, the plant and table like objects would be merged together, and the plant abstract object would grow into the table.
This process would then be repeated for other 2 merged objects, though in different ways. The last one with the red bear head and the “beaten frowns after” would be given a unique treatment, in that it would transform it into that of a book.
Now with the abstract and merged objects done, they were organised into the corresponding folders and work then began on the canvas that the final piece would be displayed. Though the work would be kept relatively simple since all that would be needed for the canvas section is to upscale it, give it a more organic like appearance, and then have a few tendrils attaches to it going upward as to make it look like it is attached to the back of the brain environment.
Now with the assets done, the task was updated on the trello board, which whilst I was behind in terms of time table progress, I was still happy with how much I was able to progress in terms of the 360 video.
I would then begin the process of creating a separate audio file for what would happen when the abstract objects would merge into each other throughout the video. This sound would have to be both distinctive and loud enough as so that the viewer would immediately know that something of significance was going on within the scene.
Once more, audacity would be my main program of choice with how familiar I am with the programs UI along with having using it extensively in the past with audio experimentation. There would be 2 separate files for merging. One would be used for when 2 abstract objects became a merged object, and the other would be used when the 3 merged objects were colliding with the canvas to create the finished piece at the end.
The first would be created via modifying an existing sound from the original 1996 Pokémon red & green titles, specifically MissingNo’s battle cry, along with a pluck generated from within audacity itself. The second would be created out of the audio that will be used for the main body of the video, just pitch pent up, and with added plucks from within audacity itself. After testing and tinkering with the audio for an hour, these were the results that I was able to come up with.
Overall, I am quite proud of the results, the object collision one can be played whilst the main body audio is still playing, but loud enough as to give off a distinctive feeling for when the collision occurs, and to give the user plenty of time to look around and find the objects in question that are merging.
The other sound plays towards the end, and does isolate the rest of the audio, the main body would slowly fade out as the audio would play in, giving an angelic conclusion to the end of the video, as the user watches the abstract art piece be physically created before them onto the canvas. With the task now done, I updated the trello board, and moved onto the next task.
Progress would slow slightly over the Christmas period, but I was still able to get some smaller parts done around the project, such as making the brain on par with the rest of the objects, flipping its norms, as well as getting the final painting ready that will be used for the end of the scene.
I decided to make the brain slightly transparent, as so that when doing the general sequence for the video, the whole ordeal would be much easier to do and will be much easier to place in the objects and animate them accordingly, rather than having to constantly go in and outside the confined space of the environement.
It would be around this time that I decided to tone down my expectations slightly, primarily due to personal family issues, so some aspects such as textures would not be able to be done for the sake of time.
I got to work on making the painting that would appear during the last few seconds of the video. Making a duplicate of the 3 merged objects and positioned them accordingly to get a rough idea of what the final piece would look like.
Paint tool SAI 2 would be my choice in making the painting, as it is a program that I am familiar with, and have a growing skill level in. I wanted to make the painting had the same abstract design as one of Ivan seals paintings, with no form of corelation in making out what the objects were, and the interpretation of the painting be entirely up to the likes of the user in question, as was the case for each of Ivan’s paintings.
Each of the objects were drawn and then adjusted into a position I believed would fit with the design that I was going for. I then made the background a faint purple to dark blue gradient, and added a light to the bottom merged object to give it a more unique flare to it.
Each of the objects were then given a little bit of shading and a unique layer filter to give them that slight abstract like feel to them, before taking it into another program, and giving the whole piece a more oil painting like appearance, which gave me this image as the final result for the piece.
With the image done, the rest of the animation process could be done, making the objects spin around the central area that the camera would be sitting on. Though animating was proving to be extremely tedious, as despite looking up countless tutorials, I could not get the audio files I had created to play along side the animation within Maya, so I had to essentially use the story board that I had created as a rough guide in terms of time.
This resulted in the final product having a big of odd movement and timing which I tried to adjust as best as I could, though some parts still ended up being off by a few frames though there was only so much that could be done when no tutorial could provide an adequate result for the solution at hand.
Due to the laptop I have not being able to properly render, I had to once more improvise on the spot through making a camera that aims around the scene. It won’t be as involved as a proper VR experience, but I will make the camera slowly aim around the scene in hopes of mimicking someone looking around the environment. I set the focal point of the camera to 18, as to make the scene look extremely large from the perspective of the user.
Conclusion
Overall, this experience within emerging technologies, has taught me many things thought out the journey in creating the final product, from considering various idea’s around a unique concept, to considering potential ethical concerns, to seeing how I was able to make the final product even with the limitations of my hardware.
If I was given an opportunity to do this over again but with all the knowledge I had acquired, I would definitely try to push myself to go for a more ambitious project, and see it through all the way to the end, rather than having to take several idea’s out due to personal constraints.
References
Artshighsky (2011) Ivan Seal Available Online: Ivan Seal – Wikipedia [Accessed 26/10/2023]
vvmtest (2019) The Caretaker – Everywhere At The End Of Time – Stage 1-6 (Complete) Available Online: https://youtu.be/wJWksPWDKOc?si=L9UrbFaZPndT9h1p [Accessed 02/11/2023]
Adam Debrowski (2020) How does the Brain Control Eyesight? Available Online: What Part of the Brain Controls Vision? (allaboutvision.com) [Accessed 09/11/2023]
Maria Emfietzoglou (N/A) Thalamus – What is it, Location, Function and More Available Online: Thalamus: What Is It, Location, Function, and More | Osmosis [Accessed 09/11/2023]
Jason Coles (2021) What causes motion sickness in VR, and how can you avoid it? Available Online: What causes motion sickness in VR, and how can you avoid it? | Space [Accessed 16/11/2023]
Image References
Ivan Seal (2016) An abstract art piece by the name “Allchav tart” by Ivan Seal [Photograph] Available Online: Ivan Seal | Paintings [Accessed 26/10/2023] Fig.01
u/hsndsomejsck (2020) An abstract art piece by the name “eptitranxisticemestionscers descending” by Ivan Seal [Photograph] Available Online: Stage 5, Ivan Seal, Oil on Canvas, 2018 : Art (reddit.com) [Accessed 26/10/2023] Fig.02
N/A (2016) A side view of the human brain with various parts ladled as to point out where the brain functions are [photograph] Available Online: Brain side view | Side view illustration of the brain. Credi… | Flickr [Accessed 08/11/2023
(note, this reference is for the unedited image, and does not contain the additional annotations added around the image by myself in MsPaint)
Ivan Seal (2012) An abstract art piece by the name “Bleach flik spitchudeeinmimowf” by Ivan Seal [photograph] Available Online: Ivan Seal | Paintings [accessed 15/11/2023] Fig.04
Ivan Seal (2016) An abstract art piece by the name “Vavax clacker” by Ivan Seal [photograph] Available Online: Ivan Seal | Paintings [accessed 15/11/2023] Fig.04
Ivan Seal (2016) An Abstract art piece by the name “garden swears beneath the gardener” [photograph] Available Online: Ivan Seal | Paintings [accessed 15/11/2023] Fig.04
Ivan Seal (2016) An abstract art piece by the name “beaten frowns after” by Ivan Seal [photograph] Available Online: Ivan Seal | The Caretaker Wiki | Fandom [accessed 10/12/2023]
Video References
FortuneEight (2023) G1 – Post Awareness Confusions – Sample Guide + info Available Online: https://youtu.be/XNyZNOTFQqg?si=JK1dxRwKnZV28HWl [accessed 03/11/2023] Fig.03
EnderWarden (2022) Everywhere At The End Of Time Organ Segment Available Online: https://youtu.be/a6abCYgu5G8?si=BfZlIIm2kccLiRbG [accessed 29/11/2023]
Music References
Everywhere at the end of time R1 – Place in the world fades away – Layer 2 (The Caretaker). Added reverb and reduced speed by 3%
Leave a Reply