Small Simplicityhttps://cocoaaa.github.io/2021-02-23T00:00:00-08:00Understanding Intelligence from Computational PerspectiveWelcome!2021-02-19T00:00:00-08:002021-02-19T00:00:00-08:00Hayley Songtag:cocoaaa.github.io,2022-03-08:<div id="chartdiv" height="400"></div>
<div id="mapdiv" height="600"></div>
<p>I'm a Computer Science PhD student at USC's <a href="https://vimal.isi.edu/">VIMAL</a> interested in how we understand observations from multiple modalities (e.g. images, audio signals and written texts), and how we extract and build representations of the semantics that is invariant across the multimodal observations.</p>
<p>Before starting my PhD, I studied Mathematics …</p>Short note on coarse-graining2021-02-23T00:00:00-08:002021-02-23T00:00:00-08:00Hayley Songtag:cocoaaa.github.io,2021-02-23:articles/2021/02/23/short-note-on-coarse-graining<p>One of the axioms in Shannon's information theory is that (Shannon's) entropy satisfies coarse-graining property:</p>
<figure>
<img src="/images/it/coarse-graining-dedeo.png" alt="coarse-graining-dedeo" width="60%" />
<figcaption> While reading Information Theory for Intelligent People by S.DeDeo
</figure>
<p>This property is closely related to the conditional probabilities.
In communication -- regardless of the types of agents involved, eg. between the people over a phone …</p>Basic concepts in measure theory2020-02-23T00:00:00-08:002020-02-23T00:00:00-08:00Hayley Songtag:cocoaaa.github.io,2020-02-23:articles/2020/02/23/basic-concepts-in-measure-theory<h2>Measure</h2>
<p><img alt="orbanz-1-2" src="images/orbanz-1-2.png">
- Intuition: roughly a measure is an integral as a function of its region
</p>
<div class="math">$$ \mu(A) = \int_{A} dx ~~\text{or,} ~~~\mu(A) = \int_{A} p(x) dx $$</div>
<p>
For example, in geometric case, <span class="math">\(\mu(A)\)</span> can be interpreted as a (physical) length (if <span class="math">\(A\)</span> is one dimensional), mass (if <span class="math">\(A …</span></p>KL Divergence2020-02-22T00:00:00-08:002020-02-22T00:00:00-08:00Hayley Songtag:cocoaaa.github.io,2020-02-22:articles/2020/02/22/kl-divergence<h2>Resource</h2>
<ul>
<li><a href="https://tinyurl.com/uta23v5">Lec on VAE</a>, by Ali Ghodsi: This lecture motivates KL Divergence as the measurement of difference in the average information content of two random varialbes, whose distributions are <span class="math">\(p\)</span> and <span class="math">\(q\)</span> in in the article.</li>
<li><a href="https://en.wikipedia.org/wiki/Kullback–Leibler_divergence">Wiki</a>: It clears up different terminologies that are (misused) to refer to the KL …</li></ul>Use `Make` for Reproduible Research2020-02-22T00:00:00-08:002020-02-22T00:00:00-08:00Hayley Songtag:cocoaaa.github.io,2020-02-22:articles/2020/02/22/use-make-for-reproduible-research<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h2 id="Basics-of-make-for-Reproducible-Research">Basics of <code>make</code> for Reproducible Research<a class="anchor-link" href="#Basics-of-make-for-Reproducible-Research">¶</a></h2>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<ul>
<li><p>A research project ca be seen as a tree of dependencies</p>
<ul>
<li>the report depends on the figures and tables</li>
<li>the figurese and tables depend on the data and the analysis scripts used to process this data</li>
</ul>
</li>
<li><p>Make is a tool for creating output files from their dependencies through pre-specified rules</p>[Paper] Data Analysis with Latent Variable Models2020-02-17T00:00:00-08:002020-02-17T00:00:00-08:00Hayley Songtag:cocoaaa.github.io,2020-02-17:articles/2020/02/17/paper-data-analysis-with-latent-variable-models<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h1 id="Data-Analysis-with-Latent-Variable-Models---Blei,-2014">Data Analysis with Latent Variable Models - <a href="https://www.annualreviews.org/doi/pdf/10.1146/annurev-statistics-022513-115657">Blei, 2014</a><a class="anchor-link" href="#Data-Analysis-with-Latent-Variable-Models---Blei,-2014">¶</a></h1><h2 id="Reading-Purpose">Reading Purpose<a class="anchor-link" href="#Reading-Purpose">¶</a></h2><p>Q: Why am I reading this?<br>
A: To understnad the motivation behind latent variable models
In particular, I want to be clear about the relations among: Probablistic Graphical Model vs. Latent Variable Model vs. Bayesian Inference. They all come up in a very similar setting, but what exactly are they?</p>Bayesian Data Analysis for dummies like me2020-02-01T00:00:00-08:002020-02-01T00:00:00-08:00Hayley Songtag:cocoaaa.github.io,2020-02-01:articles/2020/02/01/bayesian-data-analysis-for-dummies-like-me<h2>Explaining physical phenomenon consistent with observations</h2>
<p>Bayesian data analysis is a way to iteratively building a mathemtical description of a physical phenomenon of interest using observed data. </p>
<h2>Setup</h2>
<p>Bayesian inference is a method of statistical inference in which Bayes' Theorem is used to update the probability for a hypothesis (<span class="math">\(\theta …</span></p>Multimodal Distribution in Image or Text domain2020-02-01T00:00:00-08:002020-02-01T00:00:00-08:00Hayley Songtag:cocoaaa.github.io,2020-02-01:articles/2020/02/01/multimodal-distribution-in-image-or-text-domain<p>Q: What does "multimodal distribution" mean in computer vision literature (eg. image-to-image translation)?</p>
<p>While reading papers on conditional image generation using generative modeling (eg. <a href="https://tinyurl.com/s5drg9c">"Toward Multimodal Image-to-Image Translation"</a> by Zhu et al (NIPS 2017)), I wasn't clear what was meant by "one-to-many mapping" between input image domain and output image …</p>Stochastic Thinking: Predictive non-determinism2020-01-20T00:00:00-08:002020-01-20T00:00:00-08:00Hayley Songtag:cocoaaa.github.io,2020-01-20:articles/2020/01/20/stochastic-thinking-predictive-non-determinism<p>MIT 6.0002 Lec4: Stochastic Thinking
- <a href="https://www.youtube.com/watch?v=-1BnXEwHUok">YOUTUBE</a>
- <img alt="predictive-nondeterminism" src="images/predictive-nondeterminism.png"></p>
<p>Often confusing categorization of a mathematical model:
- <a href="https://tinyurl.com/sxg4ejt">SE</a>
- NB: in CS, people often use "deterministic" to mean non-randomized. This causes confusion:
> "Determinism" means non-randomized. But, "Non-determinism" does <strong>not</strong> mean "randomized".
- Determinism vs. Non-Determinism
- ...? vs. stochastic/random
- a stochastic (or random) process means, </p>
<div class="highlight"><pre><span></span><span class="err">&lt …</span></pre></div>How to read a paper2020-01-16T00:00:00-08:002020-01-16T00:00:00-08:00Hayley Songtag:cocoaaa.github.io,2020-01-16:articles/2020/01/16/how-to-read-a-paper<ul>
<li>Ref: <a href="https://tinyurl.com/teh4dhg">medium</a></li>
</ul>
<h2>Before you start</h2>
<p>Q: <strong>"why are you reading this?"</strong></p>
<ul>
<li>Write it down where you can see it while reading the paper<ul>
<li>Your purpose/goal of reading may change later. You will have a different experience then. </li>
</ul>
</li>
<li>Is there a clear answer for this question? If not, you probably …</li></ul>Let's blog with Pelican2020-01-11T00:00:00-08:002020-01-11T00:00:00-08:00Hayley Songtag:cocoaaa.github.io,2020-01-10:articles/2020/01/10/lets-blog-with-pelican<h2>Makefile</h2>
<ol>
<li>
<p><code>make html</code>: generates output html files from files in <code>content</code> folder using
development config file</p>
<ul>
<li><code>make regenerate</code>: do <code>make html</code> with "listening" to new changes</li>
<li>vs. <code>make publish</code>: similar to <code>make html</code> except it uses settings in <code>pulishconf.py</code></li>
</ul>
</li>
<li>
<p><code>make devserver</code>: (re)starts a http server in the <code>output …</code></p></li></ol>OSMNX Basics2020-01-02T00:00:00-08:002020-01-02T00:00:00-08:00Hayley Songtag:cocoaaa.github.io,2020-01-02:articles/2020/01/02/osmnx-basics<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<ul>
<li>Related <a href="https://community.topcoder.com/longcontest/?module=ViewProblemStatement&rd=17036&pm=14735">topcoder</a> competition</li>
</ul>
</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">Linear Regression2020-01-01T00:00:00-08:002020-01-01T00:00:00-08:00Hayley Songtag:cocoaaa.github.io,2020-01-01:articles/2020/01/01/linear-regression<h2>Todo:</h2>
<ul>
<li>[ ] Set up the problem</li>
<li>[ ] Simple housing problem</li>
<li>[ ] Frequentist view - minimize the squared-loss</li>
<li>[ ] Probablistic view - Discriminative classifier to model conditional distribution</li>
<li>[ ] Interactive example using <code>holoviews</code> or <code>ipywwidgets</code></li>
</ul>Cool Chart tool: `amcharts`2019-12-31T00:00:00-08:002019-12-31T00:00:00-08:00Hayley Songtag:cocoaaa.github.io,2019-12-31:articles/2019/12/31/cool-chart-tool-amcharts<p>While making a visualization for my part whereabouts for the <a href="#">front page</a> of this blog, I came across this easy-to-use visualization examples using <code>amcharts</code>. Initially, I wanted to use Google Earth Studio but it required me to import country boundaries (in KML files) as well as time to learn new …</p>Demo on CIFAR10 dataset2019-12-05T00:00:00-08:002019-12-05T00:00:00-08:00Hayley Songtag:cocoaaa.github.io,2019-12-05:articles/2019/12/05/demo-cifar10<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h1 id="CNN-example-from-official-PyTorch">CNN example from official PyTorch<a class="anchor-link" href="#CNN-example-from-official-PyTorch">¶</a></h1><ul>
<li>dataset: CIFAR10</li>
<li>02-05-2019 (Tue)</li>
</ul>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h2 id="Imports-and-Path-setup">Imports and Path setup<a class="anchor-link" href="#Imports-and-Path-setup">¶</a></h2>
</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">rasterio basics2019-11-01T00:00:00-07:002019-11-01T00:00:00-07:00Hayley Songtag:cocoaaa.github.io,2019-11-01:articles/2019/11/01/rasterio-basics<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h1 id="Gdal/Rasterio-Playground">Gdal/Rasterio Playground<a class="anchor-link" href="#Gdal/Rasterio-Playground">¶</a></h1><p>Date: 11-16-2018 (Frdiay)</p>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<ol>
<li><p>Good gdal documentation on the outputs of <code>gdalinfo</code> and <code>transform</code> matrix</p>
<ul>
<li><a href="http://download.osgeo.org/gdal/workshop/foss4ge2015/workshop_gdal.html#__RefHeading__5901_1333016408">here</a></li>
</ul>
</li>
<li><p>Rasterio resources</p>
<ul>
<li><a href="">1</a></li>
<li><a href="">2</a></li>
</ul>
</li>
</ol>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>Caution: it's not recommended to import osgeo.gdal and rasterio in the same kernel.</p>
</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">Basics of Python Image Library (PIL)2019-08-01T00:00:00-07:002019-08-01T00:00:00-07:00Hayley Songtag:cocoaaa.github.io,2019-08-01:articles/2019/08/01/basics-of-python-image-library-pil<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h2 id="Python-Imaging-Library-(PIL)">Python Imaging Library (PIL)<a class="anchor-link" href="#Python-Imaging-Library-(PIL)">¶</a></h2><ul>
<li>Date: 2020-03-01 (Sun)</li>
</ul>
</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">Interactive demo of the sigmoid activation function2019-07-01T00:00:00-07:002019-07-01T00:00:00-07:00Hayley Songtag:cocoaaa.github.io,2019-07-01:articles/2019/07/01/interactive-demo-of-the-sigmoid-activation-function<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h2 id="Load-libraries">Load libraries<a class="anchor-link" href="#Load-libraries">¶</a></h2>
</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">Xarray Introduction2019-07-01T00:00:00-07:002019-07-01T00:00:00-07:00Hayley Songtag:cocoaaa.github.io,2019-07-01:articles/2019/07/01/xarray-introduction<p>This is my sketchpad to understand xr.DataArray and xr.Dataset constructors.</p>Introduction to Planet API2019-01-10T00:00:00-08:002019-01-10T00:00:00-08:00Hayley Songtag:cocoaaa.github.io,2019-01-10:articles/2019/01/10/introduction-to-planet-api<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h1 id="Planet-API-Introduction">Planet API Introduction<a class="anchor-link" href="#Planet-API-Introduction">¶</a></h1><ul>
<li><a href="https://github.com/planetlabs/notebooks/blob/master/jupyter-notebooks/data-api-tutorials/planet_data_api_introduction.ipynb">Ref</a></li>
</ul>
</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">Interactive demo of precision-recall metric2018-09-01T00:00:00-07:002018-09-01T00:00:00-07:00Hayley Songtag:cocoaaa.github.io,2018-09-01:articles/2018/09/01/interactive-demo-of-precision-recall-metric<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h2 id="Load-libraries">Load libraries<a class="anchor-link" href="#Load-libraries">¶</a></h2>
</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">Git refspec2017-01-01T00:00:00-08:002017-01-01T00:00:00-08:00Hayley Songtag:cocoaaa.github.io,2017-01-01:articles/2017/01/01/git-refspec<p>Resource: <a href="https://git-scm.com/book/en/v2/Git-Internals-The-Refspec">git-book</a></p>
<h2>Git Remotes</h2>
<blockquote>
<p>Remote repositories: versions of your project that are hosted on the internet</p>
</blockquote>
<h3><code>git remote -v</code></h3>
<p>Let's say I cloned a repository from some repository, for instance
<code>git@github.com:cocoaaa/dip.git</code>, by running:</p>
<div class="highlight"><pre><span></span><span class="n">git</span> <span class="n">clone</span> <span class="n">git</span><span class="nd">@github</span><span class="o">.</span><span class="n">com</span><span class="p">:</span><span class="n">cocoaaa</span><span class="o">/</span><span class="n">dip</span><span class="o">.</span><span class="n">git</span>
</pre></div>
<p>Then, in this cloned …</p>