Pic-a-Daily Serverless Workshop

As you might know, Guillaume Larforge and I have a Pic-a-Daily Serverless Workshop. In this workshop, we build a picture sharing application using Google Cloud serverless technologies such as Cloud Functions, App Engine, Cloud Run and more.

Image for post
Image for post

We recently added a new service to the workshop. In this blog post, I want to talk about the new service. I also want to talk about Eventarc and how it helped us to get events to the new service.

Previous architecture

This was the previous architecture of Pic-a-Daily:


I got around to updating my Knative Tutorial from Knative v0.16.0 to the latest Knative Serving v0.18.0release and Knative Eventing v0.18.1release.

In this short blog post, I want to outline a couple of minor issues I encountered during Knative v0.18.0 upgrade. Note that I skipped v0.17 altogether, some of these changes might have happened in that release.

Istio Installation

The biggest change I encountered is how Istio is installed for Knative. In previous releases, I simply pointed to the yaml files in the latest Istio version in third_party folder of Knative Serving.

In the latest release, the third_party folder does not include Istio versions anymore. Through this commit, I learned that net-istio install scripts is the way to install Istio. …


Introduction

We recently announced a new feature, Events for Cloud Run for Anthos, to build event driven systems on Google Kubernetes Engine (GKE). In the announcement, we also stated that the solution is based on open-source Knative.

In this blog post, I want to further explain the relationship between this new feature and Knative. I also want to convince you that our solution is an easier way to deploy Knative compliant event consuming services on Google Cloud.

TLDR: Events for Cloud Run for Anthos is Knative Eventing packaged and simplified for Google Cloud.

Sure, you can still create a GKE cluster, install Istio, install Knative on top, and deploy services reading events with Knative Eventing on your own. However, it will take you much longer to set everything up, not to mention the maintenance nightmare to keep track of all versions and dependencies. …


Challenges in connecting services

When I think about my recent projects, I probably spent half of my time coding new services and the other half in connecting services. Service A calling Service B, or Service C calling an external service and using the result to feed into another Service D.

Connecting services is one of those things that ‘should be easy’ but in reality, it takes a lot of time and effort. You need to figure out a common connection format for services to use, make the connection, parse the results, and pass the results on. …


My colleague Felipe Hoffa recently published a blog post titled Get started with BigQuery and dbt, the easy way. More specifically, he showed how to install dbt in Google Cloud Shell, configure it and manually run it to create a temporary dataset in BigQuery. This is great for testing dbt + BigQuery but how do you run this kind of setup in production?

dbt documentation states that Running dbt in production simply means setting up a system to run a dbt job on a schedule, rather than running dbt commands manually from the command line.

Cloud Shell is just a temporary VM in the cloud and not suitable for production workloads. One obvious solution is to create a dedicated VM, install dbt and have some kind of cron job on that VM to run dbt on a schedule. This will work but who wants to maintain a VM? Not to mention, you need to pay for the VM per second even when dbt is not running. It’s wasteful. …


Image for post
Image for post

I finally got around to updating my Knative Tutorial from Knative v0.14.0 to the latest Knative v0.16.0 release. Since I skipped v0.15.0, I'm not sure which changes are due to v0.15.0 vs. v0.16.0. Regardless, there have been some notable changes that I want to outline in this blog post. This is not meant to be an exhaustive list. Feel free to let me know in the comments if there are other notable changes that I should be aware of.

Sugar Controller

Once I updated to v0.16.0, I realized the Broker injection into the default namespace did not work. Turns out, there's a new Sugar Controller you need to install to have Broker injection. Installing Knative page has more details and my setup scripts have been updated to install this controller. …


.NET for Google Cloud Functions (Alpha)

I spoke at many .NET conferences over the last 3–4 years and one of the top requests I always received was: When will .NET be supported on Cloud Functions?

Unfortunately, I didn’t have a good answer for a while. That all changed last month with the following tweet from Jon Skeet from our C# team:

Image for post
Image for post

.NET support is coming to Cloud Functions and it is currently public alpha. You need to sign up here to gain access.

In this blog post, I want to give the .NET support of Cloud Functions a try and see how it works. Along the way, we’ll also learn about Functions Framework that powers the .NET …


.NET Core 3.1 was released on December 3rd, 2019 and is a LTS release, supported for three years.

In Google Cloud, you could already deploy .NET Core 3.1 containers in Cloud Run (see cloud-run-dotnetcore-31) and also in App Engine flexible environment with a custom runtime.

We recently extended .NET Core 3.1 support in a couple of ways:

  1. Cloud Shell now supports .NET Core 3.1.
  2. App Engine flexible environment runtime now supports .NET Core 3.1.

.NET Core 3.1 in Cloud Shell

Inside Cloud Shell, you can see the latest 3.1.301 version:

Create an ASP.NET Core MVC app:

$ dotnet new mvc -o HelloWorldAspNetCore Running 'dotnet restore' on HelloWorldAspNetCore/HelloWorldAspNetCore.csproj... Determining projects to restore... Restored /home/atameldev/HelloWorldAspNetCore/HelloWorldAspNetCore.csproj (in 221 ms). …


Motivation

When I started working from home in mid-March, I was totally obsessed with COVID-19 news. I was constantly checking number of cases and news from the UK (where I currently live) and from Cyprus (where I’m originally from). It took me a couple of weeks to realize how unproductive this was. I started limiting myself to check for news once a day. This definitely helped me to regain sanity and productivity but it was manual.

Then, in late March, I read about COVID-19 public dataset program on BigQuery and Lak Lakshmanan’s Analyzing COVID-19 with BigQuery post where he shows how to query and plot data using this public dataset. …


In this post, I want to talk about an event-driven image processing pipeline that I built recently using Knative Eventing. Along the way, I’ll tell you about event sources, custom events and other components provided by Knative that simply development of event-driven architectures.

Requirements

Let’s first talk about the basic requirements I had for the image processing pipeline:

  1. Users upload files to an input bucket and get processed images in an output bucket.
  2. Uploaded images are filtered (eg. no adult or violent images) before sending through the pipeline.
  3. Pipeline can contain any number of processing services that can be added or removed as needed. For the initial pipeline, I decided to go with 3 services: resizer, watermarker, and labeler. The resizer will resize large images. The watermarker will add a watermark to resized images and the labeler will extract information about images (labels) and save it. …

About

Mete Atamel

Developer Advocate at Google

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store