In [1]:
!date

Sat Sep 21 01:44:38 CDT 2019


What's up and welcome to

# day 19 stalkers!!¶

Let's jump in.

First everything below the next line was written before today and might have been revised to make this semi appropriate for a post (not that there was anything especially naughty in it before, just cleaning it up a bit).

Let me give some context. I write a fair bit outside of writing these amazing blog post for you, my stalker(s). I write to chronicle things that are happening in life and what you'll soon see, to understand some crapy code crap that I don't yet understand.

Writing is the way I interact with the thoughts in my head. Sometimes I use pen and paper, sometimes I tickle the keys but it's the one thing I do to understand stuff.

I like to do it. Writing that is. Also as you have probably figured out, I'm pretty phenomenal at it. That's why you keep coming back, don't you?

Creep.

Anyway. Below, one night I felt like I was learning a friggin ton of stuff and I ran into a problem where the author didn't address how they got a bunch of this data into this file to make a graph (oh my me settle down, I've included a graph at the bottom of this post but it only appears if you read everything first). I realized I was going to have to manually edit this text file to display as the author has it displayed in their book and then had a double realization on top of that.

Manual labor on a computer? No flipping way.

So below is me stopping to digest everything I had learned up until that point (the point at which I got stuck), reviewing what I learned and then slowly trying to figure out if I could automate the creation of JavaScript files with what I know how to do in Python and even automate the start of a graphing project like the one I was working on here.

In short: This is the raw workflow from that night. The scene from my evil laboratory

# MUAUAHAHAHAHA¶

enjoy.

I have so many resources stock piled in just my twitter messages to myself alone that there is something there.

So I read something kind of reassuring the other day. It said if you don't really know what to build, just copy stuff.

That sounds like pretty good advice and also a great argument against building my own things.

I guess one tends to mess around with what one is familiar with or comfortable doing.

Sometimes exploring into the unknown can be a little scary.

Tonight I worked on some really cool things and had some pretty cool ideas.

I kept working through this Data Vis with JS book I have and I'm starting to learn something along with a little JS.

JS is super confusing. but not hard to figure out. I'm finding bugs slowly (after putting them in there)

However, I probably shouldn't follow this book linearly. That is, in order.

I think that might be a huge problem with people trying to "understand" how to "understand" code. There's no real right way but it is mainly trying to understand things you don't quite understand and then working your way backwards.

### Things I've Learned so far in the book¶

• How to use the inspector to find errors in js. it works like that.
• js arrays/data structures are kind of confusing. They seem almost inconsistent at times. I feel like there are so many brackets and I don't understand what the use is. things seem to work multiple ways and don't other times.
• completing something and having code work feel like a huge breakthrough because of something doesn't show up when you have errors - nothing appears in the browser. so suddenly a picture of a graph shows up and badda bing badda boom - I'm a Data Science and Visualization Analysis Engineer (future title someday).
• also that I need to skip some parts and also work on the beginning parts in my own way.

Also I need to stop drinking soda. Ugh.

The section I want to work on particularly is the first part and automating certain steps in data organization for the javascript graphs.

the reason for this is because the author has some data they are using for the first few graphs but it's not even organized for the tutorial they have. so I'm going to organize it.

In [1]:
# the simplest javascript data structure I've seen looks something like a Python list, just like this.

js_list = [
[1,'some data point'],
[2, 'another data point'],
[3, 'last data point I promise']
]

In [5]:
# next I want to see if I can add that list to a file as a string but save the file as a js file

print(str(js_list))

[[1, 'some data point'], [2, 'another data point'], [3, 'last data point I promise']]

In [13]:
f = open("js_list_test.js", "w+")

In [14]:
ls

app.js          flotr2.min.js    linegraph        test.js
bar_color.html  js/              move_this.ipynb  untitled.html

In [15]:
f.close()

In [17]:
f = open("js_list_test.js","a")

In [18]:
f.write("""console.log("test this");""")

Out[18]:
25
In [19]:
cat js_list_test.js


okay this is a little weirder than I thought.

In [27]:
with open("test_js_list.js", "w") as file:

In [21]:
cat test_js_list.js

console.log("test this")
In [31]:
with open("test_js_list.html", "w+") as html_file:
html_file.write("""<html>
<title>some title here</title>
<script type="text/javascript" src="./test_js_list.js"></script>
</html>""".strip())

In [29]:
cat test_js_list.html

<html>
<title>some title here</title>
<script type="text/javascript" src="./test_js_list.js"</script>
</html>

yooo that just worked.

let's go.

so now I want to see if I can get a list to show up in a js file and print it to console.

In [36]:
with open("test_js_list.js", "w") as js_file:
js_file.write("js_list = " + str(js_list) + "\nconsole.log(js_list);")

In [37]:
with open("test_js_list.html", "w+") as html_file:
html_file.write("""<html>
<title>some title here</title>
<script type="text/javascript" src="./test_js_list.js"></script>
</html>""".strip())


holy crap this is actually working.

this is actually working.

In [1]:
import numpy as np

In [9]:
co2_dataset = np.loadtxt('co2_annmean_mlo.txt', dtype="float")

In [10]:
co2_dataset

Out[10]:
array([[1.9590e+03, 3.1597e+02, 1.2000e-01],
[1.9600e+03, 3.1691e+02, 1.2000e-01],
[1.9610e+03, 3.1764e+02, 1.2000e-01],
[1.9620e+03, 3.1845e+02, 1.2000e-01],
[1.9630e+03, 3.1899e+02, 1.2000e-01],
[1.9640e+03, 3.1962e+02, 1.2000e-01],
[1.9650e+03, 3.2004e+02, 1.2000e-01],
[1.9660e+03, 3.2138e+02, 1.2000e-01],
[1.9670e+03, 3.2216e+02, 1.2000e-01],
[1.9680e+03, 3.2304e+02, 1.2000e-01],
[1.9690e+03, 3.2462e+02, 1.2000e-01],
[1.9700e+03, 3.2568e+02, 1.2000e-01],
[1.9710e+03, 3.2632e+02, 1.2000e-01],
[1.9720e+03, 3.2745e+02, 1.2000e-01],
[1.9730e+03, 3.2968e+02, 1.2000e-01],
[1.9740e+03, 3.3018e+02, 1.2000e-01],
[1.9750e+03, 3.3111e+02, 1.2000e-01],
[1.9760e+03, 3.3204e+02, 1.2000e-01],
[1.9770e+03, 3.3383e+02, 1.2000e-01],
[1.9780e+03, 3.3540e+02, 1.2000e-01],
[1.9790e+03, 3.3684e+02, 1.2000e-01],
[1.9800e+03, 3.3875e+02, 1.2000e-01],
[1.9810e+03, 3.4011e+02, 1.2000e-01],
[1.9820e+03, 3.4145e+02, 1.2000e-01],
[1.9830e+03, 3.4305e+02, 1.2000e-01],
[1.9840e+03, 3.4465e+02, 1.2000e-01],
[1.9850e+03, 3.4612e+02, 1.2000e-01],
[1.9860e+03, 3.4742e+02, 1.2000e-01],
[1.9870e+03, 3.4919e+02, 1.2000e-01],
[1.9880e+03, 3.5157e+02, 1.2000e-01],
[1.9890e+03, 3.5312e+02, 1.2000e-01],
[1.9900e+03, 3.5439e+02, 1.2000e-01],
[1.9910e+03, 3.5561e+02, 1.2000e-01],
[1.9920e+03, 3.5645e+02, 1.2000e-01],
[1.9930e+03, 3.5710e+02, 1.2000e-01],
[1.9940e+03, 3.5883e+02, 1.2000e-01],
[1.9950e+03, 3.6082e+02, 1.2000e-01],
[1.9960e+03, 3.6261e+02, 1.2000e-01],
[1.9970e+03, 3.6373e+02, 1.2000e-01],
[1.9980e+03, 3.6670e+02, 1.2000e-01],
[1.9990e+03, 3.6838e+02, 1.2000e-01],
[2.0000e+03, 3.6955e+02, 1.2000e-01],
[2.0010e+03, 3.7114e+02, 1.2000e-01],
[2.0020e+03, 3.7328e+02, 1.2000e-01],
[2.0030e+03, 3.7580e+02, 1.2000e-01],
[2.0040e+03, 3.7752e+02, 1.2000e-01],
[2.0050e+03, 3.7980e+02, 1.2000e-01],
[2.0060e+03, 3.8190e+02, 1.2000e-01],
[2.0070e+03, 3.8379e+02, 1.2000e-01],
[2.0080e+03, 3.8560e+02, 1.2000e-01],
[2.0090e+03, 3.8743e+02, 1.2000e-01],
[2.0100e+03, 3.8990e+02, 1.2000e-01],
[2.0110e+03, 3.9165e+02, 1.2000e-01],
[2.0120e+03, 3.9385e+02, 1.2000e-01],
[2.0130e+03, 3.9652e+02, 1.2000e-01],
[2.0140e+03, 3.9865e+02, 1.2000e-01],
[2.0150e+03, 4.0083e+02, 1.2000e-01],
[2.0160e+03, 4.0424e+02, 1.2000e-01],
[2.0170e+03, 4.0655e+02, 1.2000e-01],
[2.0180e+03, 4.0852e+02, 1.2000e-01]])
In [11]:
no_floats_dataset = np.format_float_positional(co2_dataset[:,:])

# this actually works pretty nicely. the data set is in scientific notation

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-11-770352789099> in <module>
----> 1 no_floats_dataset = np.format_float_positional(co2_dataset[:,:])
2
3 # this actually works pretty nicely. the data set is in scientific notation

1130                               fractional=fractional, trim=trim,
1133
1134

TypeError: only size-1 arrays can be converted to Python scalars

So all the above stuff didn't do anything as you can see but some how the data is printed out nicer below. I still can't figure out how this worked.

But I'm cool with it.

In [18]:
co2_dataset[0:3,:]

Out[18]:
array([[1.9590e+03, 3.1597e+02, 1.2000e-01],
[1.9600e+03, 3.1691e+02, 1.2000e-01],
[1.9610e+03, 3.1764e+02, 1.2000e-01]])
In [13]:
# so I want to see if I can convert it but first I don't need the last column

cut_co2_dataset = co2_dataset[:,:2]

# this slices off far column

In [20]:
cut_co2_dataset.shape # testing shape of array after column slicerooni

Out[20]:
(60, 2)
In [15]:
cut_co2_dataset[0,1] # taking samples for good measure

Out[15]:
315.97
In [16]:
cut_co2_dataset[0,0] # another sample, here here

Out[16]:
1959.0
In [17]:
cut_co2_dataset[0:10,:] # not sure why it's not displaying in non scientific notation now.

Out[17]:
array([[1959.  ,  315.97],
[1960.  ,  316.91],
[1961.  ,  317.64],
[1962.  ,  318.45],
[1963.  ,  318.99],
[1964.  ,  319.62],
[1965.  ,  320.04],
[1966.  ,  321.38],
[1967.  ,  322.16],
[1968.  ,  323.04]])

Alright so now I will convert the array to a python list with a super lazy but awesome method .tolist(). Thanks numpy creators.

In [21]:
py_list_co2_data = cut_co2_dataset.tolist()
py_list_co2_data[0:10][:]

Out[21]:
[[1959.0, 315.97],
[1960.0, 316.91],
[1961.0, 317.64],
[1962.0, 318.45],
[1963.0, 318.99],
[1964.0, 319.62],
[1965.0, 320.04],
[1966.0, 321.38],
[1967.0, 322.16],
[1968.0, 323.04]]

Going to try to write to files now.

In [22]:
with open("linegraph.html", "w+") as html_file:
html_file.write("""<html>
<title>some title here</title>
<script type="text/javascript" src="./linegraphData.js"></script>
<script type="text/javascript" src="./linegraph.js"></script>
</html>""".strip())

with open("linegraphData.js", "w+") as js_data_file:
js_data_file.write("linegraphData = " + str(py_list_co2_data))

with open("linegraph.js", "w+") as js_file:
js_file.write("console.log(linegraphData)")


# future changes for above script¶

• add the word "var" in front of array name
• add the script that imports pertinent js library (example: )
• add semi colon to end of array
• don't forget to add the container to the html file.
• make sure to read all of the code.

Good thing to review your code once in awhile. (once in a while being the operative phrase) Let's test that those statements above made files..

In [25]:
ls line*

linegraphData.js  linegraph.html  linegraph.js


Yeah buddy. Lettttttsss gooooo. Putting a break in between this here markdown and the shtuff below for future reference. Also I'm going to write right here that everything below this line is the start of another project

added from the future (oooooooooo): didn't realize that the below html would have actually just prevented all of the errors I just faced in making the graph work with the code that I used and wrote in the file from above.

alright so I have the basic build of automating the html file creation of this.

<html>
<title>some title here</title>
<script type="text/javascript" src="./linegraphData.js"></script>
<script type="text/javascript" src="./linegraph.js"></script>
<script type="text/javascript" src="./flotr2.min.js"></script>
<div id="chart" style="width:600px; height:300px"></div>
</html>

In [78]:
# which a function could maybe look like this

def make_new_html_graph_file(name_of_html_file,name_of_js_file=None, name_of_data_file=None):
"""makes html file ready for rendering new graph that is loaded from js data file"""

new_html_file_name = name_of_html_file + ".html"

with open(new_html_file_name, "w+") as html_file:
html_file.write("""
<html>
<title>some title here</title>
<script type="text/javascript" src="./linegraphData.js"></script>
<script type="text/javascript" src="./linegraph.js"></script>
<script type="text/javascript" src="./flotr2.min.js"></script>
<div id="chart" style="width:600px; height:300px"></div>\n</html>
""")

In [79]:
make_new_html_graph_file("fart")

In [80]:
ls

app.js               flotr2.min.js     linegraph.js       test_js_list.js
bar_color.html       js/               move_this.ipynb    untitled.html
co2_annmean_mlo.txt  linegraph         test.js
fart.html            linegraphData.js  test_js_list.html

In [81]:
cat fart.html

<html>
<title>some title here</title>
<script type="text/javascript" src="./linegraphData.js"></script>
<script type="text/javascript" src="./linegraph.js"></script>
<script type="text/javascript" src="./flotr2.min.js"></script>
<div id="chart" style="width:600px; height:300px"></div>
</html>


was this all just to write "cat fart" in this blog post?

why yes. Yes it was.

alright alright alright here's the graph I promised from above.