research-tech | James Colliander

2i2c + Berkeley

Thu, 21 Oct 2021 09:00:19 +0000

Open Toolchain

Fri, 27 Aug 2021 00:00:00 +0000

`Open Toolchain`

Canadian Open Data Society Board meeting

2021-08-27

Idea

People, with the right data, tools and resources, can do good things working together.

James Colliander

Professor of Mathematics: UBC
Cofounder: 2i2c, Callysto, Syzygy, Crowdmark
Past Director (2016-2021): PIMS

Background

PIMS: Distributed Mathematical Sciences Institute

Inspirations

Data 8 at Berkeley

Partners

Idea: Jupyter for 🇨🇦

Development History

gantt
title Timeline
section Syzygy
23 universities, 10 research teams: 2020-02-25, 25d
40,000 users : 2021-08-15, 25d
25,000 users : 2020-01-01, 25d
16,000 users : 2019-08-22, 25d
8,000 users : 2017-08-24, 25d
Syzygy service announcement : 2017-03-15, 25d
PIMS + Compute Canada Agreement : 2016-06-02, 25d
Compute Canada Discussions : 2015-12-03, 2016-06-01
Jessica Hamrick's Post : 2015-03-24, 25d
JupyterHub for 🇨🇦 idea : 2015-02-01, 90d
section Callysto
CanCode 2.0 🇨🇦 $1.5M : 2019-10-15, 25d
Callysto content creation : 2018-05-01, 2018-08-31
CanCode 1.0 🇨🇦 $1.5M : 2018-01-30, 25d
Callysto Launched: 2017-10-01, 25d

Examples

Lessons Learned

Research and education transforming
Demand can be met with cloud strategy
Scaling requires infrastructure

Sustainable business model?

`2i2c`

International Interactive Computing Collaboration

Syzygy, Callysto and 2i2c

Tue, 25 Feb 2020 23:00:00 +0000

National Scale Interactive Computing

Wed, 22 Aug 2018 19:32:16 +0000

(Invited post on Jupyter blog: https://blog.jupyter.org/national-scale-interactive-computing-2c104455e062)

This is an invited post from Jim Colliander, Professor of Mathematics at UBC and Director of the Pacific Institute for the Mathematical Sciences.¹

In 2017, the Pacific Institute for the Mathematical Sciences (PIMS), in partnership with Compute Canada and Cybera, launched Syzygy, a cloud-hosted interactive computing platform that delivers JupyterHub deployments for universities across Canada.

Syzygy has been used by over 16,000 students at 20 universities. The main results of the Syzygy experiment so far are:

Demand for interactive computing is ubiquitous² and growing strongly at universities.
The Jupyter ecosystem is an effective way to deliver interactive computing.
A scalable, sustainable, and cost-effective interactive computing service for universities is needed as soon as possible.

Demand for interactive computing

Both research and teaching at universities are adapting to major societal changes driven by explosions in data and computational tools. New educational programs that prepare students to think computationally are emerging, while research strategies are changing in ways that are more open, reproducible, collaborative, and interdisciplinary. These transformations are inextricably linked and are accelerating demand for interactive computing. The Syzygy experiment has shown that using Jupyter in educational programs drives interest in using Jupyter for research (and vice versa). For example, students in mathematics, statistics, and computer science collaborated with a researcher from St. Paul’s Hospital in Vancouver using Syzygy to identify new pathways to prevent death from sepsis. Research communities typically need access to deeper computational resources and often span multiple universities, but the common thread is the need to expand access to interactive computing.

A map of JupyterHub deployments deployed by Syzygy.

Technical milestone achieved

The Syzygy project has demonstrated that it’s possible to deploy tools for interactive computation at a national scale rapidly and efficiently using an entirely open source technology stack. Students, faculty and staff across Canada use Syzygy to access Jupyter through their browsers with their university single-sign-on credentials. The JupyterHubs range from a “standard” configuration to bespoke environments with specially curated tools and data integrations. This richness is possible because of the architecture of the Syzygy and Jupyter projects and the flexibility of the underlying cloud resources. As a case-study, Syzygy demonstrates that the Jupyter community has achieved a significant technical milestone: interactive computing can be delivered at national scale using cloud technologies.

Service level requirements

The validation that interactive computing can be technically delivered at national scale prompts universities to ask a variety of questions. Can interactive computing service be delivered robustly? How will users be supported? What are the uptime expectations? What is the data security policy? How is privacy protected? Can the robustness of the service be clarified in a service level agreement? Syzygy, as an experimental service offered to universities at no charge and without a service level agreement, does not properly address these questions. To advance on their education-research-service mission and address growing demand, universities need a reliable interactive computing service with a service level agreement.

What’s next?

How should universities address their needs for interactive computing over the next five years? Right now, universities are following two primary approaches:

🙏 Ad hoc: faculty figure out how to meet their own needs for interactive computing; IT staff deploys JupyterHub on local or commercial cloud servers; this approach gives universities control over their deployments and hardware, though requires time and expertise that many may not have.
🎩 Use a cloud provider’s service: Google Colab, Amazon Sagemaker, Microsoft Azure Notebooks, IBM Watson Studio; this approach allows universities to quickly launch interactive computing services, with a loss of flexibility and some risks by becoming reliant upon a particular vendor’s closed-source and proprietary software.

These approaches are not sustainable over the long term. If universities all deploy their own JupyterHub services, many will need technical expertise they do not currently have and will involve a significant duplication of effort. If universities rely on hosted cloud notebook services, the reliance on proprietary technology will impair their ability to switch between different cloud vendors, change hardware, customize software, etc. Vendor lock-in will limit the ability of universities to respond to changes in price for the service. Universities will lose agility in responding to changes in faculty, staff, and student computing needs.

There is a third option that addresses the issues with these two approaches and generates other benefits for universities:

🤔 Form an interactive computing consortium: universities collaborate to build an interactive computing service provider aligned with their missions to better serve their students, facilitate research, and avoid risks associated with vendor lock-in.

To retain control over their interactive computing stacks, avoid dependence⁴ on cloud providers, and accelerate the emergence of new programs, universities should work together to deploy interactive computing environments in a vendor-agnostic manner. This might take the form of a consortium — an organization dedicated to serving the needs of universities through customized shared infrastructure for interactive computing. The consortium would also ensure that universities will continue to play a leadership role in the development of the interactive computing tools used for education and research.

The Syzygy experiment confirmed that growing demand for interactive computation within universities can be supplied with the available technologies advanced by the Jupyter open source community. In the coming year, we aim to build upon the success of the Syzygy experiment and seed an initial node of a consortium in Canada with the intention of fostering a global network of people invested in advanced interactive computing. If you are interesting in partnering, please get in touch! See this Jupyter Community Forum post to continue the discussion.

The author gratefully acknowledges feedback on this piece from Ian Allison, Lindsey Heagy, Chris Holdgraf, Fernando Perez, and Lindsay Sill.
Interactive computing needs have been identified in agriculture, applied mathematics, astronomy, chemistry, climate science, computer science, data science, digital humanities, ecology, economics, engineering, genomics, geoscience, health sciences, K-12 education, neuroscience, political science, physics, pure mathematics, statistics, and sociology.
Relying on commercial cloud vendors to provide the interactive computing service for universities risks recreating the problems associated with scientific publishing that emerged with the internet.

Artificial Intelligence as a Service: Text Analysis

Fri, 03 Jun 2016 19:11:25 +0000

The computing infrastructure as a service offered by Amazon Web Services (AWS) may have originally been conceived as a resource to support their own electronic commerce business but other striking applications emerged. Netflix toppled Blockbuster and transformed the way we consume video atop the AWS infrastructure. Dropbox changed the way we store and share digital resources by building on AWS. My company Crowdmark leverages AWS to store and serve images of student work for evaluation by graders and further analysis. Recently, the Google Cloud Vision API, the IBM Watson Developer Cloud and the Receptiviti.ai API started offering artificial intelligence as a service available for purchase like a utility. These resources may form the foundation for a new era of technological metamorphosis.

Experimenting with Text Analysis

I wondered whether text analysis might generate useful insights for Crowdmark or other applications so I performed some experiments. I ran the texts from inaugural addresses by four presidents of the United States through some text analysis tools I found online. The IBM Watson Tone Analyzer uses “cognitive linguistic analysis methods” to measure the emotional tone in text. The Linguistic Inventory Word Count (LIWC) is a computer text analysis tool developed and psychometrically validated by James Pennebaker. Receptiviti.ai is a Toronto-based startup that offers text analysis as a service based on LIWC and other technology.

The texts

The tools

Screen captures of the results appear below.

Conclusion

The IBM Watson Tone Analyzer results are exposed within an intuitive and interactive user interface. The results do not correspond well with my own emotional response reviewing the videos or reading these speeches. Based on these experiments, I am not convinced that Tone Analyzer will generate useful insights into the emotional characteristics of text. Based on what I observed, there appear to be too few dimensions of emotional tone generated by Tone Analyzer for it to drive improvements to the dialogue between instructors and students. There may be a rich superset of output measurements not exposed in this free demonstration.

The LIWC results exposed through this free sample analysis are a small collection of the many dimensions measured by LIWC. It is not easy to glean insights into the speaker’s personality or their emotional tone based on the reports externalized in this free demo.

Short text descriptions of personality traits of the speaker emerged in the results from Receptiviti.ai. I found the text descriptions interesting but with limited precision in describing the speakers. I’m curious to know whether the personality decription accuracy increases with larger text samples from the same speaker.

Files of various types (text, images) may now be sent to increasingly sophisticated online analysis engines poised and ready to extract data and return insights to the sender. What will your robot assistant read for you tomorrow?

Experiments with IBM Watson Tone Analyzer

IBM Watson Tone Analyzer

Reagan Experiment

As a first experiment, I copied the text from the January 20, 1981 Inaugural Address of President Ronald Reagan (video) and pasted it into the IBM Watson Tone Analyzer. Here are the overview results:

Moving the mouse over portions of the text reveals the results of tonal analysis of the highlighted paragraph.

Tonal dimensions can be highlighted and sentences can be ranked based on tonal strength. For example, here is the sentence ranked highest for Disgust.

Clinton Experiment

As a second experiment, I processed the text from the January 20, 1993 Inaugural Address of President Bill Clinton (video) with Tone Analyzer. Here are the overview results:

Tone Analyzer reports that Clinton’s speech is most dominated by Fear.

Kennedy Experiment

For a third experiment, I chose the text from January 20, 1961 Inaugural Address of President John Kennedy (video):

Eisenhower Experiment

For the final experiment, I processed the text from January 21, 1957 Inaugural Address of President Dwight Eisenhower (video):

Overview Comparisons using IBM Watson Tone Analyzer

Reagan 1980

Clinton 1993

Kennedy 1961

Eisenhower 1957

Experiments with LIWC

LIWC

Reagan via LIWC
Clinton via LIWC
Kennedy via LIWC
Eisenhower via LIWC

Experiments with Receptiviti Personality Insights

Receptiviti.ai

Reagan via Receptiviti.ai
Clinton via Receptiviti.ai
Kennedy via Receptiviti.ai
Eisenhower via Receptiviti.ai

Larry Guth Colloquium: Unexpected Applications of Polynomials in Combinatorics

Mon, 11 Feb 2013 20:01:01 +0000

My former colleague Larry Guth (now at MIT) visited us recently and gave a beautiful colloquium talk. The Department has recently deployed a video streaming service so we are able to share Larry’s talk with the world. We look forward to sharing other videos in the future.

Here is the video:

Your browser does not support the <video> tag.

Unexpected applications of polynomials in combinatorics

by Larry Guth | MIT Time: 16:10 (Wednesday, Jan. 23, 2013) Location: BA6183, Bahen Center, 40 St George St Abstract: In the last five years, several hard problems in combinatorics have been solved by using polynomials in an unexpected way. In some cases, the proofs are very short, and I will present a complete proof in the lecture. One of the problems is the joints problem. Given a set of lines in $R^3$, a joint is a point that lies in three non-coplanar lines. Given $L$ lines in $R^3$, how many joints can there be? Another problem is the distinct distance problem in the plane. If P is a set of points in the plane, the distance set of $P$ is the set of all distances from one point of $P$ to another. If $P$ is a set of $N$ points in the plane, how small can the distance set of $P$ be? The proofs involve studying a set of points in a vector space by finding a polynomial of controlled degree that vanishes at the points, and then using the geometry of the zero-set to understand the combinatorial properties of the points. The goal for the talk is to give an overview of this new method.

Web Course Platform Prototyped by Toronto Mathematician

Thu, 09 Feb 2012 19:35:49 +0000

Suppose you’ve recorded a one hour lecture onto video and taken digital photographs of each blackboard used during the presentation. The video and photograph files contain metadata with time stamps: we know the time when each frame of video and photograph was captured. How should these files be blended to create a browser-based environment for studying the ideas in the lecture? My colleague Dror Bar-Natan has prototyped a remarkable software platform combining this data to support online courses and seminars.

The system is built atop MediaWiki (the software that runs Wikipedia) and allows metadata, such as comments and photographs, to be correlated in time to an embedded video stream. Comments can be attached to each frame of video. The system enables remote audience members to participate in the seminar frame-by-frame using their power to comment. In this way, the wClips system goes beyond playback by empowering the remote audience of the future to contribute to the past presentation.

The system was recently used to supplement a paper Dror is co-authoring with Zsuzsanna Dansco in the wClips seminar. Each section of the paper was discussed in detail in a seminar lecture captured on video. Each blackboard in the seminar series was photographed. Time links near the photographs move the video stream to the video time when that blackboard image was created. The sizes of the different elements can be changed by sliding the red line. In the final version of the paper, there will appear hyperlinks in each section pointing at the associated seminar presentation. Clicking on the photographs reveals another layer of comments highlighting the elements appearing on the blackboard.

Dror gave a video introduction describing how to operate the system and produced a one page guide. (The lighting is not so good.) Zsuzsanna’s lecture is also a nice demonstration of wClips.

The wClips system runs on a linux server using open source software so is inexpensive to deploy. Since it is based on MediaWiki, it has the potential to scale to support a massive audience. Getting the video content out of the camera and into the online course system involves some work but could be streamlined with the right scripts.

Fingertip Search Using University Library Resource

Fri, 06 Jan 2012 19:30:27 +0000

UTORvpn

Members of the University of Toronto have access to a fantastic library. There are great resources like the Oxford English Dictionary, MathSciNet and Web of Knowledge and a whole lot more. To access these resources while away from campus, members of the UofT community can use the virtual private network service UTORvpn. That site tells you how to install the VPN service on your computer. When you turn on the VPN, your computer can access restricted library resources even though it is not located on the University network. On my computer, the active and inactive VPN service is indicated like this:

[caption id=“attachment_711” align=“alignleft” width=“241” caption=“Active VPN”][/caption]

[caption id=“attachment_712” align=“alignleft” width=“236” caption=“Inactive VPN”][/caption]

Standard Search

Suppose you wanted to look up the definition of the word syzygy. You could use google to get some information or you could drill down through the University of Toronto library looking for the Oxford English Dictionary to eventually find a more definitive result.

Researchers are busy. So, many will merely use google since the drilling down through the library takes too much time.

Customized Search at your fingertips

UofT’s restricted databases can be made available at your fingertips. Here is an example using Firefox. Right click (or control-mouse) the search field on the OED to reveal the option Add a Keyword for this Search

I chose to use the keyword oed.

Now, I can type into the URL bar, oed WORDTOLOOKUP, and I get the OED search results instantly.

The same thing can be done with AMS MathSciNet author search and other search engines.

Mathematics and the Library at the University of Toronto

Wed, 06 Apr 2011 05:47:23 +0000

Overview

UTL has built, and is in the process of expanding, a suite of web-based tools that have the potential to revolutionize the research driven by scholars at the University of Toronto. Some of these tools are designed to open the library’s archival system to new content streams generated within the University’s daily research activities. Other tools provide new methods to collect, share and discuss library resources enabling new ways to collaborate and develop research ideas. The technologies underpinning these advances are already in place or soon to be deployed. The next main challenge is the principal obstruction: we need to stimulate the researchers to use them. My view is that they won’t unless they see a payoff for their research goals.

New tools and their potential

A quick summary of web-based resources provided by UTL organized along four key categories:

Archival
Scholarship
Collaboration
Broadcast

Archival

The library’s tspace is an online repository for storing content generated by the University of Toronto. This resource is a bit like arXiv but localized to UofT and it is open to a wider variety of content types (movies, scans of hand-written notes, etc.) than arXiv. Once uploaded to tspace, the content is permanently archived, given an immutable web location address and is strategically exposed to web search engines. All PhD theses produced here end up on tspace. The library wants scholars to upload preprint and postprint versions of published papers to tspace. Notes should be uploaded to tspace enabling researchers to later make references to the ideas emerging from seminars, etc. The remarkable feature of tspace is that it enables UofT researchers to directly influence the Library’s collection. However, the success of the tspace repository is limited without the active participation by UofT researchers who are plenty busy with other tasks.

Scholarship

The library has piloted and will soon deploy BibApp, a web gateway organizing library content around the scholars at UofT who produced it. (Here is my page inside the test server.) BibApp’s database of citations is automatically filled (with UTL sometimes doing massive amounts of data cleaning) using content streams from resource amalgamators like Scholar’s Portal. However, BibApp also leverages the expertise of the scholar’s who appear inside BibApp by allowing them to edit and improve BibApp’s content about their scholarship. The application shows scholars information about the copyright restrictions of their published works:

Those references which appear green can be enriched with postprint and preprint versions of the published work. BibApp will be intertwined with tspace so that uploads into tspace can take place within the BibApp pages.

Collaboration

Peter Clinton reports to me that the library will soon provide storage space enabling faculty to build and share citation collections through zotero. Zotero provides rapid collection of reference data on published works, preprints on arXiv, web pages, etc. and stores the collected data in a database that is accessible through the web. Each citation is stored inside a container into which other data can be developed. This technology allows for precise exchanges of ideas to take place among the references shared by a group of scholars. This can be extremely useful for a research group led by a UofT faculty member or for a broader group of researchers with similar interests.

Broadcast

The library has invested in broadcast video streaming technology and built MyMedia. MyMedia is UofT’s version of youtube. Videos generated by UofT faculty can be uploaded in their native format. The MyMedia server converts and makes available the video content in various formats so that the content can be watched on iPhone, Android, through a web browser, etc. The library has also made recording and screen capture technologies available to UofT faculty so it is easy to generate video content for instruction and research purposes.

But, will people use this technology?

The technology is in place. However, the experience of libraries across the world over the past ten years building these types of tools spawns a common observation: to make it really work requires changing the behavior of people.

UTL and Toronto Math share some resources. These resources will be deployed in pilot programs to encourage and make it easier for math faculty to take advantage of the Library’s technologies. The shared resources should be spent strategically to create short term examples that mathematicians can recognize as valuable within their research plans. We need examples that show how these tools enhance the research potential of faculty-led research groups to inspire adoption of the technology into the daily work flow of researchers at UofT. Updates to follow….

research-tech | James Colliander

2i2c + Berkeley

Open Toolchain

Open Toolchain

Idea

James Colliander

Background

PIMS: Distributed Mathematical Sciences Institute

Inspirations

Jessica Hamrick’s Post

Data 8 at Berkeley

Partners

Idea: Jupyter for 🇨🇦

Development History

Examples

Lessons Learned

Sustainable business model?

2i2c

Syzygy, Callysto and 2i2c

National Scale Interactive Computing

Demand for interactive computing

Technical milestone achieved

Service level requirements

What’s next?

Artificial Intelligence as a Service: Text Analysis

Experimenting with Text Analysis

Conclusion

Experiments with IBM Watson Tone Analyzer

Reagan Experiment

Clinton Experiment

Kennedy Experiment

Eisenhower Experiment

Overview Comparisons using IBM Watson Tone Analyzer

Experiments with LIWC

Experiments with Receptiviti Personality Insights

Larry Guth Colloquium: Unexpected Applications of Polynomials in Combinatorics

Unexpected applications of polynomials in combinatorics

Web Course Platform Prototyped by Toronto Mathematician

Fingertip Search Using University Library Resource

UTORvpn

[caption id=“attachment_712” align=“alignleft” width=“236” caption=“Inactive VPN”][/caption]

Standard Search

Customized Search at your fingertips

Mathematics and the Library at the University of Toronto

Overview

New tools and their potential

Archival

Scholarship

Collaboration

Broadcast

But, will people use this technology?

`Open Toolchain`

`2i2c`