An AI agent is a computer program or system that can perceive its environment, process information, and make decisions or take actions to achieve specific goals. AI agents are designed to mimic some aspects of human intelligence and behavior, allowing them to operate autonomously or semi-autonomously in their environment.

The connection between AI agents and LLMs lies in the way they interact. AI agents use LLMs as a reasoning engine to determine which actions to take and in which order to take them. Instead of using an LLM directly, you can use an agent and give it access to a set of tools to deliver a sequence based on your intended task. This approach enables intelligent cost-effective interactions between the agent and an LLM.

How to build an AI Agent easily and quickly?

I am now using some open source tools to build my own AI Agent, which involves two major open source projects: BotSharp/ UI and LLamaSharp. I will introduce each step in detail in this blog and keep it updated. The steps introduced here are about a way to develop and run locally, so they will involve the operation of some configuration files and the explanation of some C# code. Because the environment used here is .NET, this technical blog is suitable as a reference for developers who already have a certain foundation in .NET/ C#. After the entire project is configured, you will probably see a running screenshot like this:

Let’s start recording every step of the operations

We use a business scenario demonstration officially provided by BotSharp, Pizza Restaurant, to show the specific configuration steps. First, we pull the source code from Github to local. This step requires cloning two open source projects.

git clone https://github.com/SciSharp/BotSharp
cd BotSharp
dotnet run --project .\src\WebStarter

git clone https://github.com/SciSharp/BotSharp-UI
cd BotSharp-UI
npm install
npm run dev

If there is no reference to BotSharp.Plugin.LLamaSharp.csproj in your local WebStarter project, please add the project reference manually. And add BotSharp.Plugin.LLamaSharp to PluginLoader.Assemblies in appsetting.json. Finally, you need to install the LLamaSharp backend package in WebStarter. You can choose to install the CPU version or the CUDA version. After both the front-end and back-end projects are started normally, you can visit http://localhost:5015/ to continue the operation. You can log in using the default administrator account. This account has been placed in the local configuration file.

The second step is to configure compatible llama 2 model settings. You need to add an LLM Provider. Before doing this, you need to download the model file. The following is the reference code for LLM Provider configuration. You need to replace the configuration of llama-sharp.

"LlmProviders": [
  {
    "Provider": "azure-openai",
    "Models": [
      {
        "Name": "gpt-35-turbo",
        "ApiKey": "",
        "Endpoint": "https://gpt-35-turbo-chat.openai.azure.com/",
        "Type": "chat",
        "PromptCost": 0.0015,
        "CompletionCost": 0.002
      },
      {
        "Name": "gpt-35-turbo-instruct",
        "ApiKey": "",
        "Endpoint": "https://gpt-35-turbo-text.openai.azure.com/",
        "Type": "text",
        "PromptCost": 0.0015,
        "CompletionCost": 0.002
      }
    ]
  },
  {
    "Provider": "llama-sharp",
    "Models": [
      {
        "Name": "llama-2-7b-guanaco-qlora.Q2_K.gguf",
        "Type": "chat"
      }
    ]
  }
]

After modifying the appsettings.json file, you need to restart the BotSharp project for the update to take effect. Next, you can see the LLamaSharp plug-in in the Plugins interface. It is still in the Disabled status. Click Install to enable this plug-in. Regarding the operating principle of BotSharp plug-in, you can refer to the official documentation to learn more. In order to run the Pizza Restaurant example, you also need to install the PizzaBot plugin.

After installing these two plug-ins, click the menu Agents, and you will see that there are several more Agents here. These Agents are provided by the corresponding plug-ins. You can also make your own Agent plug-in to meet your business needs. Here we can see agents of Ordering, Order Inquiry and Payment. Different Agents will perform corresponding processing according to different requests from users.

There is one last configuration left to support the Llama 2 model. This configuration can be modified in Agent details. Update the Model of LLM Config’s Provider to your configuration. Model is the name of the model file you just downloaded.

Okay, great work has been done. You can now test PizzaBot. Return to the Agents interface and click the Chat Test button under Ordering agent. If you are using LLamaSharp’s CPU Backend, you will encounter the problem of slow response speed. Switching to the GPU version will be much faster.

OK, so far, you can use .NET to run an LLM Agent of the Llama model locally. BotSharp is an open source, extensible Agent design and management tool. If you are interested in learning more about this project, please go to Github to save it and star it.

Building AI Agent in Llama 2 and .NET 6 was originally published in SciSharp STACK on Medium, where people are continuing the conversation by highlighting and responding to this story.

In the field of artificial intelligence, people’s expectations for Agents are growing day by day. Whenever a new open source tool or product based on Agent appears, it can trigger heated discussions, such as the previous AutoGPT. For those who are interested in Agent, I recommend a paper, which comprehensively introduces the architecture of Agent and is of great value for understanding the overall situation of Agent.

This paper explains in detail the concept, development history and recent research hotspots of Agent. In addition to these basic knowledge, I think the most valuable part is that it summarizes the architecture of Agent based on large language model (LLM), allowing us to design our own Agent according to a certain standard paradigm.

This article of mine mainly explains the construction strategy of Agent based on LLM from two key aspects: designing the Agent architecture to better utilize the capabilities of LLM, and how to give the Agent the ability to complete different tasks.

In terms of Agent architecture design, this paper proposes a unified framework, including Profile module, Memory module, Planning module and Action module.

Profile module

Define and manage the characteristics and behaviors of Agent roles. It contains a series of parameters and rules that describe various attributes of the Agent, such as roles, goals, abilities, knowledge, and behavior. These properties determine how the agent interacts with the environment, how it understands and responds to tasks, and how it makes decisions and plans. This module proposes three Agent role generation methods, including LLM generation method, data set alignment method and combination method.

1. LLM generation method: Use a large language model to automatically generate the personal characteristics of the agent, such as age, gender, personal preferences and other background information. The specific method is: first set the composition rules of agents and clarify the attributes that agents in the target population should have; then specify several manually created seed configuration files as examples; and finally use language models to generate a large number of agent configuration files. This approach can quickly generate configuration files in batches, but the resulting agents may lack detail due to the lack of precise control.

2. Data set alignment method: The agent’s profile information is obtained from real-world population data sets, such as by extracting population survey data and organizing it into a natural language description. This can make the agent behavior more realistic and credible and accurately reflect the attribute distribution of the real population. But it requires reliable large-scale data set support.

3. Combination method: Use real data sets to generate a part of key agents to ensure that they reflect the laws of the real world; then use the LLM generation method to supplement a large number of other agents to expand the number of agents. This not only ensures the authenticity of the agents, but also achieves a sufficient number of agents, allowing the system to simulate more complex social interactions. Careful configuration file design is the basis for building an effective proxy system.

Memory module

It plays an important role in the Agent system, which stores and organizes information obtained from the environment to guide future actions.

Structurally, a memory module usually contains two parts: short-term memory and long-term memory. Short-term memory temporarily stores recent perceptions, and long-term memory stores important information for retrieval at any time.

In terms of format, memory information can be expressed in natural language or encoded into vector embeddings to improve retrieval efficiency. You can also use database storage, or organize it into a structured list to represent memory semantics.

In operation, it mainly interacts with the environment through three mechanisms: memory reading, writing and reflection. Read and extract relevant information to guide actions, write and store important information, reflect and summarize insights to improve the level of abstraction.

Planning module

The main task is to help the Agent decompose complex tasks into more manageable sub-tasks and formulate effective strategies. It is roughly divided into two types, one is a plan that does not rely on feedback, and the other is a plan based on feedback.

Feedback-independent plans do not refer to post-task feedback during the formulation process and have several common strategies. For example, single-path reasoning generates plans step by step in a cascading manner. In addition, there is multi-path reasoning, which generates multiple alternative planning paths to form a tree or graph-like structure. Of course, we can also use an external planner to quickly search to find the optimal plan.

Feedback-based planning, which adjusts the plan based on feedback after task execution, is more suitable for situations where long-term planning is required. The source of feedback may come from objective feedback of task execution results, feedback based on human subjective judgment, or even feedback provided by an auxiliary model.

Action module

The responsibility is to transform abstract decisions into concrete actions. It is like a bridge that connects the Agent’s internal world and the external environment. When performing a task, consider the goal of the action, how it was generated, its scope of application, and its likely impact.

Ideally actions should be purposeful, such as completing a specific task, communicating with other agents, or exploring the environment. Actions can be generated by consulting past memory experiences or by following a preset plan. The scope of action can not only be expanded by leveraging external tools such as APIs and knowledge bases, but also requires leveraging the inherent capabilities of large language models (LLM), such as planning, dialogue, and understanding common sense.

The architecture is like the hardware of a PC, but relying solely on architectural design is not enough. We also need to give the Agent the ability to complete different tasks. These are regarded as “software” resources. Several methods are proposed in the paper, including model fine-tuning, prompt engineering, and mechanical engineering. Among them, prompt engineering is probably the most common form. The prompt word engineer we often hear is the role in this context.

The design and construction strategy of LLM-based Agent is a complex and challenging task. With the advancement of technology, I believe that more excellent AI applications will be produced through Agents in the future, and ordinary users can also create their own Agents through open source projects and become super individuals in the AI era. I hope everyone can take action as soon as possible, reserve more knowledge, and use or make their own Agent as soon as possible when the technology matures.

The design and construction strategy of LLM-based Agent is a complex and challenging task. With the advancement of technology, I believe that more excellent AI applications will be produced through Agents in the future, and ordinary users can also create their own Agents through open source projects and become super individuals in the AI era. For .NET developers, there are already some excellent frameworks to accelerate LLM system integration, such as Semantic Kernel and BotSharp. I hope everyone can take action as soon as possible, reserve more knowledge, and use or make their own Agent as soon as possible when the technology matures.

Understand the LLM Agent Orchestration was originally published in SciSharp STACK on Medium, where people are continuing the conversation by highlighting and responding to this story.

2023 is the year when LLM begins to explode. Everyone is busy integrating LLM with their existing systems and making full use of LLM’s language and reasoning capabilities. With the help of LLM, chatbots show their true intelligence. There are currently several tools that can help developers integrate LLM, such as LangChain, LlamaIndex and EmbedChain, as well as Microsoft’s Semantic Kernel and AutoGen. These frameworks can accelerate LLM application development at a certain point.

I would like to introduce to you another LLM open source framework developed by .NET, BotSharp. It is an enterprise-level LLM application construction framework written entirely in C# language. It can effectively accelerate the integration of enterprise business systems and LLM, and provides a comprehensive service from Agent. A complete set of code mechanisms from management to conversation management can truly be applied to production-level code quality, with the best programming paradigms and practices.

Let’s start with a simple introductory example to see how BotSharp connects traditional applications and LLM. This example is a pizza store’s AI customer service, which can help users create new pizza orders. Assume that you have been able to run the project and access the local Open API interface. The basic project setup code can be found here.

1. Create a Pizza Ordering Agent

After creating the agent, you will see a new folder with the AgentId. Each file inside has a different purpose:

• agent.json
  This file contains basic information recorded for the agent.
• instruction.liquid
  This file contains a description of the agent’s capabilities, telling LLM your current role and business process. This prompt is defined using the liquid template, and you can use the Hook function to dynamically set some variable values.
• functions.json
  This file is a function description of the agent. Functions are the only way for the Agent to interact with the outside world. The Agent can only access the outside world through function calls. They are defined using the JSON Schema standard.

Using files to define LLM’s Prompt and Function can be easily modified in subsequent expansions. The principle of separation of prompts and code is also implemented in the design.

2. Now you can start your chatbot via command:

dotnet run --project .\src\WebStarter\WebStarter.csproj -p SolutionName=PizzaBot

Start a conversation with the BOT in Postman

Did you notice that in the second conversation, LLM triggered a call to the function get_pizza_types. This function will correspond to the function in C# by name, and the specific function behavior can be defined and executed using C# code.

public class GetPizzaTypesFn : IFunctionCallback
{
    public string Name => "get_pizza_types";

    public async Task<bool> Execute(RoleDialogModel message)
    {
        message.ExecutionResult = "Pepperoni Pizza, Cheese Pizza, Margherita Pizza";
        message.ExecutionData = new List<string>
        {
            "Pepperoni Pizza",
            "Cheese Pizza",
            "Margherita Pizza"
        };
        return true;
    }
}

Let’s continue our conversation with Pizza Ordering:

Bot will communicate with users based on the business process we defined in the previous instruction.liquid template. After the user confirms the quantity, the get_pizza_price function will be called to calculate the total order price.

After the user confirms the order, the Bot will ask the user how to support the order. These processes are carried out completely in accordance with our process design. Meeting our business expectations.

From the above simple example, you can see that building LLM through BotSharp is very straightforward and simple. Scalability and code readability are very good. This simple example cannot demonstrate the power of BotSharp. In subsequent articles, I will introduce some of its more powerful functions to show how it can complete complex enterprise applications. The complete sample code can be found here.

In mathematics, a tensor is an algebraic object that describes a multilinear relationship between sets of algebraic objects related to a vector space. Tensors may map between different objects such as vectors, scalars, and even other tensors.

Tensor can be represented in the form of multidimensional data, and each dimension can be used to represent a specific set of features. Let’s talk about how to use C# to do basic Tensor operations. .NET itself does not have a Tensor library, we need to use a third-party open source library to do Tensor operations.

PM> Install-Package TensorFlow.NET
PM> Install-Package SciSharp.TensorFlow.Redist

How to create a Tensor?

After installing the required libraries, we can start coding and operating Tensor. First, let’s create a 1-dimensional Tensor. This creation process is similar to initializing an array in C#. The only difference is that we need to use np.array to wrap the array. Tensor can be represented by a multidimensional array structure, so Tensor and NDArray are essentially the same thing. They can be converted to each other without performance loss, because the data blocks point to the same memory area.

using static Tensorflow.Binding;
using Tensorflow.NumPy;

// Create an array in normal way
var a1 = new[] { 1, 2, 3, 4 };
// Wrap it as NDArray
NDArray nd1 = np.array(a1);
// Convert NDArray to Tensor
Tensor t1 = nd1;

// We can also initialize a Tensor from NDArray, 
// That's because there is an implicity conversion.
// Tensor t1 = np.array(a1);

Essentially, the operation on Tensor is the operation on NDArray. So in the following code snippets we can directly use NDArray instead of Tensor to do various operations in C#. The most commonly used operations on Tensor are data type conversion and dimension transformation, which can be easily done through the API provided in the TensorFlow Numpy library. We can transform the nd1 array just now into a 2x2 two-dimensional array:

var nd2 = nd1.reshape((2, 2));

// By looking at the data pointers of these two arrays, 
// you can see that they point to the same data block. 
// Therefore, the conversion of dimensions has almost no performance loss.
// nd1.buffer == nd2.buffer

// But for data type conversion, it is necessary to reallocate memory.
var nd3 = nd2.astype(np.float32);

For creational operations, we also have several commonly used initialization methods:

// Allocate a 4 elements filled with zero.
var nd4 = np.zeros(4);

// Allocate a NDArray and filled with one.
var nd5 = np.ones(4);

// Allocate a 4 double data type of memory without default value.
var nd6 = np.empty(4);

// And even an array that contains a range of evenly spaced intervals.
var nd7 = np.arange(2, 9, 2);

// You can also use np.linspace() to create an array with values that are spaced linearly 
// in a specified interval:
var nd8 = np.linspace(0, 10, num: 5);

// While the default data type is floating point (np.float64), 
// you can explicitly specify which data type you want using the dtype keyword.
var nd9 = np.ones(2, dtype: np.int64);

// Return an array of ones with the same shape and type as a given array.
nd7 = nd7.reshape((2, 2));
var nd10 = np.ones_like(nd7);

// Return an array of zeros with the same shape and type as a given array.
nd10 = np.zeros_like(nd7);

How do we know the Tensor’s shape and number of dimensions after the NDArray has been created or modified? There are 4 important properties in NDArray: ndim, shape, size and dtype

ndim will tell you the number of axes, or dimensions, of the array.
size will tell you the total number of elements of the array. This is the product of the elements of the array’s shape.
shape will display a tuple of integers that indicate the number of elements stored along each dimension of the array. If, for example, you have a 2-D array with 2 rows and 3 columns, the shape of your array is (2, 3).
dtype will tell you the data type of the array.

// shape is (2, 2)
print(nd2.shape);

// ndim is 2
print(nd2.ndim);

// dtype is int32
print(nd2.dtype);

// size is 4
print(nd2.size);

Concatenate, Sorting elements

When you want to sort an existing array, you can use np.sort() to operate, which will return a new sorted array. You can use np.concatenate()to combine two different arrays.

var arr = np.array(new[] { 2, 1, 5, 3, 7, 4, 6, 8 });
arr = np.sort(arr);

// adding more elements
var arr2 = np.array(new[] { 2, 3 });
var arr3 = np.concatenate((arr, arr2));

The above content is about the most basic data creation of Tesnor. This article will continue to be updated and add more code explanations about Tensor operations.

C# Tensor: the absolute basics for beginners was originally published in SciSharp STACK on Medium, where people are continuing the conversation by highlighting and responding to this story.

TensorFlow 2.x has three mode of graph computation, namely static graph construction (the main method used by TensorFlow 1.x), Eager mode and AutoGraph method. In TensorFlow 2.x, the official recommendation is using the Eager mode and the AutoGraph method.

The Eager mode is convenient for model development and debugging by data scientist, the code is readable and the coding efficiency is high for the machine learning engineers, but the computation performance is not as good as the static computation graph. The execution efficiency of the static graph is very high, but the code writing and readability are poor, and the program debugging is difficult.

In TensorFlow 2, eager execution is turned on by default. The user interface is intuitive and flexible (running one-off operations is much easier and faster), but this can come at the expense of performance and deployability.

AutoGraph takes into account the two major advantages of dynamic graph coding efficiency and static graph execution efficiency. Literally understood, the AutoGraph mechanism is an automatic calculation graph conversion function, which can write codes developed in accordance with dynamic graph writing rules. , Through the AutoGraph mechanism, it is converted into a static computation graph and executed in the same way as a static calculation graph, achieving the purpose of “quickly writing code and debugging code in a dynamic graph” + “efficient code execution in a static graph”.

Of course, the code and usage scenarios that the AutoGraph mechanism can convert are not unconstrained (or it will completely replace the static calculation graph and the dynamic calculation graph). The fitness of the AutoGraph mechanism requires certain coding rules and a deep understanding of AutoGraph. Perform the internal operation process of the calculation graph conversion, otherwise the conversion will fail, or an abnormal execution process that does not meet the expectations will occur.

In the next two sections, we will explain in detail the principles and coding specifications of AutoGraph’s internal operating mechanism. Next, we will use a simple example to understand how to use the AutoGraph mechanism.

This is the simplest Tensor numeric multiplication operation:

public void Run()
{
    var a = tf.constant(2);
    var b = tf.constant(3);
    var output = Mul(a, b);
    print(output);

    Console.ReadKey();
}

Tensor Mul(Tensor a, Tensor b)
{
    return a * b;
}

After running, it will output the result of multiplying the values ​​2 * 3 = 6:

tf.Tensor: shape=(), dtype=int32, numpy=6

This process is carried out in dynamic computation graph mode, namely numerical computation in Eager Mode.

In TensorFlow for .NET, we implement the AutoGraph mechanism in two ways:

Method ① Manually run the tf.autograph.to_graph() method to convert the function into a static computation graph;

Method ② Add Attribute property label AutoGraph before the function body to modify the function to realize automatic static graph conversion.

We recommend ways ② [AutoGraph] way, more flexible and convenient, the code more readable, let these two ways illustrate, respectively.

① tf.autograph.to_graph() detailed description

We can use the tf.autograph.to_graph() method to convert the dynamic calculation graph to a static calculation graph. The usage is very simple. It can be completed by adding a conversion code directly. Let’s perform the AutoGraph conversion test on the above multiplication operation.

Inject tf.autograph.to_graph() for AutoGraph conversion:

public void Run()
{
    var func = tf.autograph.to_graph(Mul);// AutoGraph
    var a = tf.constant(2);
    var b = tf.constant(3);
    var output = func(a, b);
    print(output);

    Console.ReadKey();
}

Tensor Mul(Tensor a , Tensor b)
{
    return a * b;
}

By adding a line of code var func = tf.autograph.to_graph(Mul), we can realize the calculation graph conversion of the AutoGraph mechanism, and the result of the multiplication operation remains unchanged:

tf.Tensor: shape=(), dtype=int32, numpy=6

AutoGraph operating efficiency

Generally speaking, when the computational graph model is composed of a large number of small operations, the efficiency of AutoGraph is greatly improved. If the number of model operations is small, the efficiency of AutoGraph is not greatly improved, sometimes due to the time-consuming conversion of computational graphs , Resulting in longer running time.

Let’s make a simple modification to the previous example and add a loop test to test the running time under different conditions.

Case 1# We directly increase the loop, and run the multiplication loop one million times

code show as below:

public void Run()
{
    Stopwatch sw = new Stopwatch();
    sw.Restart();
    foreach (var item in range(1000000))
    {
        var a = tf.constant(2);
        var b = tf.constant(3);
        var output = Mul(a, b);
    }
    sw.Stop();
    print("Eager Mode：" + sw.Elapsed.TotalSeconds.ToString("0.0 s"));

    sw.Restart();
    var func = tf.autograph.to_graph(Mul);//AutoGraph
    foreach (var item in range(1000000))
    {
        var a = tf.constant(2);
        var b = tf.constant(3);
        var output = func(a, b);
    }
    sw.Stop();
    print("AutoGraph Mode：" + sw.Elapsed.TotalSeconds.ToString("0.0 s"));

    Console.ReadKey();
}

Tensor Mul(Tensor a, Tensor b)
{
    return a * b;
}

The results are as follows:

Eager Mode：20.2 s
AutoGraph Mode：41.8 s

We can see that the Eager mode 20.2 s is faster than the AutoGraph mode 41.8 s (different PC configurations may have slightly different calculation results).

Case 2# We add a little operand, and then run the multiplication operation in the same loop one million times

The operand of the multiplication operation has been added and modified to 4 consecutive multiplications a * b * a * b * a * b * a * b, the code is as follows:

public void Run()
{
    Stopwatch sw = new Stopwatch();
    sw.Restart();
    foreach (var item in range(1000000))
    {
        var a = tf.constant(2);
        var b = tf.constant(3);
        var output = Mul(a, b);
    }
    sw.Stop();
    print("Eager Mode：" + sw.Elapsed.TotalSeconds.ToString("0.0 s"));

    sw.Restart();
    var func = tf.autograph.to_graph(Mul);//AutoGraph
    foreach (var item in range(1000000))
    {
        var a = tf.constant(2);
        var b = tf.constant(3);
        var output = func(a, b);
    }
    sw.Stop();
    print("AutoGraph Mode：" + sw.Elapsed.TotalSeconds.ToString("0.0 s"));

    Console.ReadKey();
}

Tensor Mul(Tensor a , Tensor b)
{
    return a * b * a * b * a * b * a * b;
}

The results are as follows:

Eager Mode：78.3 s
AutoGraph Mode：47.1 s

This time, we can see that Eager mode 78.3 s is much slower than AutoGraph mode 47.1 s (different PC configurations may have slightly different calculation results), and the efficiency advantage of AutoGraph has been reflected.

Next, let’s look at other simple code examples of AutoGraph.

Example 1: Simple conditional comparison operation

A simple way to compare the size of 2 Tensor constant values ​​and output the smaller value:

public  void  Run ()
{
    var func = tf.autograph.to_graph(Condition_Min);
    var a = tf.constant(3);
    var b = tf.constant(2);
    var output = func(a, b);
    print(output);

    Console.ReadKey();
}

// TwoInputs_OneOutput_Condition
Tensor Condition_Min(Tensor a, Tensor b)
{
    return tf.cond(a < b, a, b);
}

Run the code, the smaller value 2 is correctly output:

tf.Tensor: shape=(), dtype=int32, numpy=2

Example 2: AutoGraph conversion of Lambda expression

The AutoGraph mechanism also supports the conversion of static calculation graphs of Lambda expressions. A simple sample code is as follows:

public  void  Run ()
{
    var func = tf.autograph.to_graph((x, y) => x * y);
    var output = func(tf.constant(3), tf.constant(2));
    print(output);

    Console.ReadKey();
}

Running the code, the operation result of the Lambda multiplication expression is correctly output 6:

tf.Tensor: shape=(), dtype=int32, numpy=6

② [AutoGraph] detailed description

Another recommended AutoGraph conversion method is to add the Attribute property label [AutoGraph] in front of the function body to modify the function to realize automatic static graph conversion. This method is more convenient, the amount of code changes is less, and the program can be modified and more readable.

Let’s take a look through a few different examples.

Example 1# Floating point addition operation

This time we do an addition operation of floating-point decimals, the code is as follows:

public  void  Run ()
{
    var a = tf.constant(2.0);
    var b = tf.constant(1.5);
    var output = Add(a, b);
    print(output);

    Console.ReadKey();
}

Tensor Add(Tensor a, Tensor b)
{
    var c = a + b;
    return c;
}

After running, the program outputs the result of the addition operation:

tf.Tensor: shape=(), dtype=TF_DOUBLE, numpy=3.5

We can implement AutoGraph conversion by adding the feature tag [AutoGraph] in front of the Add() method. The code is as follows:

public  void  Run ()
{
    var a = tf.constant(2.0);
    var b = tf.constant(1.5);
    var output = Add(a, b);
    print(output);

    Console.ReadKey();
}
[AutoGraph]
Tensor Add(Tensor a, Tensor b)
{
    var c = a + b;
    return c;
}

After running, the program outputs the same result of the addition operation:

tf.Tensor: shape=(), dtype=TF_DOUBLE, numpy=3.5

Example 2# MNIST logistic regression test

Next, we will test a slightly more complicated case-MNIST logistic regression. For the specific content of MNIST logistic regression, you can refer to the example Handwritten Character Classification Logistic Regression.

At the same time, we added a timer to test whether there is a difference in time between Eager mode and AutoGraph mode.

The complete console code of MNIST logistic regression in Eager mode is as follows:

using static Tensorflow.KerasApi;
using Tensorflow;
using static Tensorflow.Binding;
using Tensorflow.Keras.Optimizers;
using System.Diagnostics;

namespace Test_Core
{
    class Program
    {
        static void Main(string[] args)
        {
            Stopwatch sw = new Stopwatch();
            sw.Restart();

            TFNET  tfnet  =  new  TFNET ();
            tfnet . Run ();

            sw.Stop();
            print("Time Last：" + sw.Elapsed.TotalSeconds.ToString("0.0 s"));
        }
    }

    class TFNET
    {
        ResourceVariable W = tf.Variable(tf.ones((784, 10)), name: "weight");
        ResourceVariable b = tf.Variable(tf.zeros(10), name: "bias");
        SGD optimizer = keras.optimizers.SGD(0.01f);
        public  void  Run ()
        {
            int training_epochs = 20000;
            int batch_size = 256;
            int num_features = 784; // 28*28
            int display_step = 1000;
            float accuracy = 0f;

            var ((x_train, y_train), (x_test, y_test)) = keras.datasets.mnist.load_data();
            (x_train, x_test) = (x_train.reshape((-1, num_features)), x_test.reshape((-1, num_features)));
            (x_train, x_test) = (x_train / 255f, x_test / 255f);

            var train_data = tf.data.Dataset.from_tensor_slices(x_train, y_train);
            train_data = train_data.repeat().shuffle(5000).batch(batch_size).prefetch(1);
            train_data = train_data.take(training_epochs);

            foreach (var (step, (batch_x, batch_y)) in enumerate(train_data, 1))
            {
                run_optimization(batch_x, batch_y);

                if (step % display_step == 0)
                {
                    var pred = logistic_regression(batch_x);
                    var loss = cross_entropy(pred, batch_y);
                    var acc = Accuracy(pred, batch_y);
                    print($"step: {step}, loss: {(float)loss}, accuracy: {(float)acc}");
                    accuracy = acc.numpy();
                }
            }

            {
                var pred = logistic_regression(x_test);
                print($"Test Accuracy: {(float)Accuracy(pred, y_test)}");
            }
        }
        Tensor Accuracy(Tensor y_pred, Tensor y_true)
        {
            var correct_prediction = tf.equal(tf.argmax(y_pred, 1), tf.cast(y_true, tf.int64));
            return tf.reduce_mean(tf.cast(correct_prediction, tf.float32));
        }
        Tensor cross_entropy(Tensor y_pred, Tensor y_true)
        {
            y_true = tf.cast(y_true, TF_DataType.TF_UINT8);
            y_true = tf.one_hot(y_true, depth: 10);
            y_pred = tf.clip_by_value(y_pred, 1e-9f, 1.0f);
            return tf.reduce_mean(-tf.reduce_sum(y_true * tf.math.log(y_pred), 1));
        }
        Tensor logistic_regression(Tensor x)
        {
            return tf.nn.softmax(tf.matmul(x, W) + b);
        }
        private void run_optimization(Tensor x, Tensor y)
        {
            using var g = tf.GradientTape();
            var pred = logistic_regression(x);
            var loss = cross_entropy(pred, y);
            var gradients = g.gradient(loss, (W, b));
            optimizer.apply_gradients(zip(gradients, (W, b)));
        }

    }
}

The program execution took 47.2 seconds, and the running result output is as follows (different PCs may have slightly different execution time and calculation results):

step: 1000, loss: 0.60991216, accuracy: 0.859375
step: 2000, loss: 0.52773017, accuracy: 0.8515625
step: 3000, loss: 0.49317402, accuracy: 0.86328125
step: 4000, loss: 0.3669852, accuracy: 0.8984375
step: 5000, loss: 0.34490967, accuracy: 0.91015625
step: 6000, loss: 0.36495364, accuracy: 0.90625
step: 7000, loss: 0.3448297, accuracy: 0.91796875
step: 8000, loss: 0.3595497, accuracy: 0.890625
step: 9000, loss: 0.4222083, accuracy: 0.87109375
step: 10000, loss: 0.3518588, accuracy: 0.90625
step: 11000, loss: 0.3254419, accuracy: 0.91015625
step: 12000, loss: 0.3375517, accuracy: 0.90234375
step: 13000, loss: 0.47696534, accuracy: 0.875
step: 14000, loss: 0.38535655, accuracy: 0.890625
step: 15000, loss: 0.4106849, accuracy: 0.875
step: 16000, loss: 0.33494166, accuracy: 0.90625
step: 17000, loss: 0.41341114, accuracy: 0.91796875
step: 18000, loss: 0.33995813, accuracy: 0.90625
step: 19000, loss: 0.32934952, accuracy: 0.9140625
Test Accuracy: 0.9166
Time Last：47.2 s

We can implement AutoGraph conversion by adding the feature tag [AutoGraph] in front of the optimizer run_optimization() method. The complete console code is as follows:

using static Tensorflow.KerasApi;
using Tensorflow;
using static Tensorflow.Binding;
using Tensorflow.Keras.Optimizers;
using System.Diagnostics;
using Tensorflow.Graphs;

namespace Test_Core
{
    class Program
    {
        static void Main(string[] args)
        {
            Stopwatch sw = new Stopwatch();
            sw.Restart();

            TFNET  tfnet  =  new  TFNET ();
            tfnet . Run ();

            sw.Stop();
            print("Time Last：" + sw.Elapsed.TotalSeconds.ToString("0.0 s"));
        }
    }

    class TFNET
    {
        ResourceVariable W = tf.Variable(tf.ones((784, 10)), name: "weight");
        ResourceVariable b = tf.Variable(tf.zeros(10), name: "bias");
        SGD optimizer = keras.optimizers.SGD(0.01f);
        public  void  Run ()
        {
            int training_epochs = 20000;
            int batch_size = 256;
            int num_features = 784; // 28*28
            int display_step = 1000;
            float accuracy = 0f;

            var ((x_train, y_train), (x_test, y_test)) = keras.datasets.mnist.load_data();
            (x_train, x_test) = (x_train.reshape((-1, num_features)), x_test.reshape((-1, num_features)));
            (x_train, x_test) = (x_train / 255f, x_test / 255f);

            var train_data = tf.data.Dataset.from_tensor_slices(x_train, y_train);
            train_data = train_data.repeat().shuffle(5000).batch(batch_size).prefetch(1);
            train_data = train_data.take(training_epochs);

            foreach (var (step, (batch_x, batch_y)) in enumerate(train_data, 1))
            {
                run_optimization(batch_x, batch_y);

                if (step % display_step == 0)
                {
                    var pred = logistic_regression(batch_x);
                    var loss = cross_entropy(pred, batch_y);
                    var acc = Accuracy(pred, batch_y);
                    print($"step: {step}, loss: {(float)loss}, accuracy: {(float)acc}");
                    accuracy = acc.numpy();
                }
            }

            {
                var pred = logistic_regression(x_test);
                print($"Test Accuracy: {(float)Accuracy(pred, y_test)}");
            }
        }
        Tensor Accuracy(Tensor y_pred, Tensor y_true)
        {
            var correct_prediction = tf.equal(tf.argmax(y_pred, 1), tf.cast(y_true, tf.int64));
            return tf.reduce_mean(tf.cast(correct_prediction, tf.float32));
        }
        Tensor cross_entropy(Tensor y_pred, Tensor y_true)
        {
            y_true = tf.cast(y_true, TF_DataType.TF_UINT8);
            y_true = tf.one_hot(y_true, depth: 10);
            y_pred = tf.clip_by_value(y_pred, 1e-9f, 1.0f);
            return tf.reduce_mean(-tf.reduce_sum(y_true * tf.math.log(y_pred), 1));
        }
        Tensor logistic_regression(Tensor x)
        {
            return tf.nn.softmax(tf.matmul(x, W) + b);
        }
        [AutoGraph]
        private void run_optimization(Tensor x, Tensor y)
        {
            using var g = tf.GradientTape();
            var pred = logistic_regression(x);
            var loss = cross_entropy(pred, y);
            var gradients = g.gradient(loss, (W, b));
            optimizer.apply_gradients(zip(gradients, (W, b)));
        }

    }
}

This time the program execution took 45.3 seconds, and the running result output is as follows (different PCs may have slightly different execution time and calculation results):

step: 1000, loss: 0.5845629, accuracy: 0.88671875
step: 2000, loss: 0.42607152, accuracy: 0.8984375
step: 3000, loss: 0.5029169, accuracy: 0.85546875
step: 4000, loss: 0.33637348, accuracy: 0.90625
step: 5000, loss: 0.4658397, accuracy: 0.875
step: 6000, loss: 0.3632791, accuracy: 0.89453125
step: 7000, loss: 0.38538253, accuracy: 0.90234375
step: 8000, loss: 0.3392726, accuracy: 0.90234375
step: 9000, loss: 0.30917627, accuracy: 0.90234375
step: 10000, loss: 0.3470266, accuracy: 0.9140625
step: 11000, loss: 0.33611432, accuracy: 0.921875
step: 12000, loss: 0.37535334, accuracy: 0.89453125
step: 13000, loss: 0.35773978, accuracy: 0.890625
step: 14000, loss: 0.35660252, accuracy: 0.8984375
step: 15000, loss: 0.23310906, accuracy: 0.9453125
step: 16000, loss: 0.3695503, accuracy: 0.88671875
step: 17000, loss: 0.31830546, accuracy: 0.90625
step: 18000, loss: 0.2850374, accuracy: 0.91796875
step: 19000, loss: 0.26065654, accuracy: 0.9375
Test Accuracy: 0.917
Time Last：45.3 s

Comparing the results of the second time, after adding the AutoGraph mechanism, the program operation efficiency is not significantly improved, and the execution time is not much different.

Through the above two examples of “MNIST Logistic Regression Example” and “One Million Multiplication Operations”, we can see that for a single operation that is time-consuming, the performance improvement brought by the AutoGraph mechanism will not be much , And when the model consists of many small operations, the efficiency of the AutoGraph mechanism is greatly improved. In the actual application process, you can comprehensively consider specific application scenarios and selectively apply the AutoGraph mechanism.

Principle of AutoGraph Mechanism

When we use the tf.autograph.to_graph() or [AutoGraph] method to convert a function into a static calculation graph, what happens inside the program?

For example, we test the following code:

public  void  Run ()
{
    var func = tf.autograph.to_graph(Add);//AutoGraph
    print("First Run:Initialization");

    var a = tf.constant(2);
    var b = tf.constant(3);
    var output = func(a, b);
    print($"Second Run:{output.numpy()}");

    output = func(output, b);
    print($"Third Run:{output.numpy()}");

    Console.ReadKey();
}

Tensor Add(Tensor a, Tensor b)
{
    foreach (var i in range(3))
        print(i);
    var c = a + b;
    print(c);
    print("tracing");

    return c;
}

After running, we will see the following results:

0
1
2
tf.Tensor 'add:0' shape=<unknown> dtype=int32
tracing
First Run:Initialization
Second Run:5
Third Run:8

Let’s analyze this process step by step. In general, two things have happened.

When we wrote the function body of Add(), nothing happened, only the signature of such a function was recorded in the C# memory stack.

When we call this function for the first time through the tf.autograph.to_graph() method, the first thing happens. The program creates a static calculation graph by analyzing the content of the Add() function body. The parsing process will track the code that executes the weight of the function, determine the Tensor type of each variable, and add the OP operator to the calculation graph according to the execution order to generate a fixed static calculation graph.

Therefore, we see that the terminal sequentially outputs the value of the loop variable i = 0 1 2, the initial content of the Tensor add operator c inside the function, and the print test string tracing at the end of the function body. Here we can also see from the content of the tensor c inside the function body that c is just an operation operator in the static calculation graph, and the initial calculation result 0 is not obtained according to the Eager mode.

When we call the function for the second time and pass in the parameter Tensor to be calculated, the second thing happens. The program automatically calls the static calculation graph just created, passes in the Tensor to be calculated, executes the calculation graph, and outputs the calculated result. In this process, the program no longer runs the internal code of the function body, and directly calls the calculation graph created in the previous step for calculations, so the print() method of many tests inside the function body will not be executed again.

Therefore, the program only outputs the second result “Second Run: 5”.

This operation is no longer in the Eager mode method, but similar to the following statement executed in TensorFlow 1.x:

using (var sess = tf.Session(tf.get_default_graph()))
{
    var result = sess.run(tf.constant(2),tf.constant(3));
}

The process of calling the function for the third time is exactly the same as that for the second time. The program still calls the original calculation graph for repeated calculations and outputs the result “Third Run: 8”.

We can see that through the AutoGraph mechanism, the execution efficiency of repeated operations can be improved dramatically.

Here I simply demonstrate the process by way of diagrams, as shown in the following figure:

Thanks for the SciSharp STACK again, bring fancy Machine Learning technology to .NET community.

TensorFlow AutoGraph for .NET was originally published in SciSharp STACK on Medium, where people are continuing the conversation by highlighting and responding to this story.

ThensorFlow is capable to handle multiple threads, and queues are powerful mechanism for asynchronous computation. If we have large datasets this can significantly speed up the training process of our models. This functionality is especially handy when reading, pre-processing and extracting in mini-batches our training data. The secret to being able to do professional and high performance training of our model is understanding TensorFlow queuing operations. TensorFlow has implemented 4 types of Queue: FIFOQueue, PaddingFIFOQueue, PriorityQueue and RandomShuffleQueue.

Like everything in TensorFlow, a queue is a node in a computation graph. It’s a stateful node, like a variable: other nodes can modify its content, In particular, nodes can enqueue new items into the queue, or dequeue existing items from the queue.

To get started with queue, let’s consider a simple example. We will create a “first in, first out” queue (FIFOQueue) and fill it with numbers. Then we’ll construct a graph that takes an item off the queue, adds one to that item, and puts it back on the end of the queue.

[TestMethod]
public void FIFOQueue()
{
	// create a first in first out queue with capacity up to 2
	// and data type set as int32
	var queue = tf.FIFOQueue(2, tf.int32);
	// init queue, push 2 elements into queue.
	var init = queue.enqueue_many(new[] { 10, 20 });
	// pop out the first element
	var x = queue.dequeue();
	// add 1
	var y = x + 1;
	// push back into queue
	var inc = queue.enqueue(y);

	using (var sess = tf.Session())
	{
		// init queue
		init.run();

		// pop out first element and push back calculated y
		(int dequeued, _) = sess.run((x, inc));
		Assert.AreEqual(10, dequeued);

		(dequeued, _) = sess.run((x, inc));
		Assert.AreEqual(20, dequeued);

		(dequeued, _) = sess.run((x, inc));
		Assert.AreEqual(11, dequeued);

		(dequeued, _) = sess.run((x, inc));
		Assert.AreEqual(21, dequeued);
        
		// thread will hang if you run sess.run(x) again
		// until queue has more element.
	}
}

Enqueue, EnqueueMany and Dequeue are special nodes. They take a pointer to the queue instead of a normal value, allowing them to change it. I first create a FIFOQueue queue of size up to 3, I enqueue two values into the queue. Then I immediately attempt to dequeue a value from it and assign it to y where I simply add 1 to the dequeued variable. Next, we start up a session and run. After we’ve run this operation a few times the queue will be empty - if we try and run the operation again, the main thread of the program will hang or block - this is because it will be waiting for another operation to be run to put more values in the queue.

FIFOQueue

Creates a queue that dequeues elements in a first-in first-out order. A FIFOQueue has bounded capacity; supports multiple concurrent producers and consumers; and provides exactly-once delivery. A FIFOQueue holds a list of up to capacity elements. Each element is a fixed-length tuple of tensors whose dtypes are described by dtypes, and whose shapes are optionally described by the shapes argument.

PaddingFIFOQueue

A FIFOQueue that supports batching variable-sized tensors by padding. A PaddingFIFOQueue may contain components with dynamic shape, while also supporting dequeue_many. A PaddingFIFOQueue holds a list of up to capacity elements. Each element is a fixed-length tuple of tensors whose dtypes are described by dtypes, and whose shapes are described by the shapes argument.

[TestMethod]
public void PaddingFIFOQueue()
{
	var numbers = tf.placeholder(tf.int32);
	var queue = tf.PaddingFIFOQueue(10, tf.int32, new TensorShape(-1));
	var enqueue = queue.enqueue(numbers);
	var dequeue_many = queue.dequeue_many(n: 3);

	using(var sess = tf.Session())
	{
		sess.run(enqueue, (numbers, new[] { 1 }));
		sess.run(enqueue, (numbers, new[] { 2, 3 }));
		sess.run(enqueue, (numbers, new[] { 3, 4, 5 }));

		var result = sess.run(dequeue_many[0]);

		Assert.IsTrue(Enumerable.SequenceEqual(new int[] { 1, 0, 0 }, result[0].ToArray<int>()));
		Assert.IsTrue(Enumerable.SequenceEqual(new int[] { 2, 3, 0 }, result[1].ToArray<int>()));
		Assert.IsTrue(Enumerable.SequenceEqual(new int[] { 3, 4, 5 }, result[2].ToArray<int>()));
	}
}

PriorityQueue

A queue implementation that dequeues elements in prioritized order. A PriorityQueue has bounded capacity; supports multiple concurrent producers and consumers; and provides exactly-once delivery. A PriorityQueue holds a list of up to capacity elements. Each element is a fixed-length tuple of tensors whose dtypes are described by types, and whose shapes are optionally described by the shapes argument.

[TestMethod]
public void PriorityQueue()
{
	var queue = tf.PriorityQueue(3, tf.@string);
	var init = queue.enqueue_many(new[] { 2L, 4L, 3L }, new[] { "p1", "p2", "p3" });
	var x = queue.dequeue();

	using (var sess = tf.Session())
	{
		init.run();

		// output will 2, 3, 4
		var result = sess.run(x);
		Assert.AreEqual(result[0].GetInt64(), 2L);

		result = sess.run(x);
		Assert.AreEqual(result[0].GetInt64(), 3L);

		result = sess.run(x);
		Assert.AreEqual(result[0].GetInt64(), 4L);
	}
}

RandomShuffleQueue

A queue implementation that dequeues elements in a random order. A RandomShuffleQueue has bounded capacity; supports multiple concurrent producers and consumers; and provides exactly-once delivery. A RandomShuffleQueue holds a list of up to capacity elements. Each element is a fixed-length tuple of tensors whose dtypes are described by dtypes, and whose shapes are optionally described by the shapes argument.

[TestMethod]
public void RandomShuffleQueue()
{
	var queue = tf.RandomShuffleQueue(10, min_after_dequeue: 1, dtype: tf.int32);
	var init = queue.enqueue_many(new[] { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 });
	var x = queue.dequeue();

	string results = "";
	using (var sess = tf.Session())
	{
		init.run();

		foreach(var i in range(9))
			results += (int)sess.run(x) + ".";

		// output in random order
		// 1.2.3.4.5.6.7.8.9.
	}
}

Queue methods must run on the same device as the queue. FIFOQueue and RandomShuffleQueue are important TensorFlow objects for computing tensor asynchronously in a graph. For example, a typical input architecture is to use a RandomShuffleQueue to prepare inputs for training a model:

• Multiple threads prepare training examples and push them in the queue.
• A training thread executes a training op that dequeues mini-batches from the queue.

This architecture simplifies the construction of input pipelines.

Download C# Example Code

From the above example, once the output gets to the point above you’ll actually have to terminate the program as it is blocked. Now, this isn’t very useful. What we really want to happen is for our little program to reload or enqueue more values whenever our queue is empty or is about to become empty. We could fix this by explicitly running our enqueue_op again in the code above to reload our queue with values. However, for large, more realistic programs, this will become unwieldy. Thankfully, TensorFlow has a solution.

TensorFlow provides two classes to help multi-threading task: tf.Coordinator and tf.QueueRunner. There two classes are designed to be used together. The Coordinator class helps multiple threads stop together and report exceptions to a main thread. The QueueRunner class is used to create a number of threads cooperating to enqueue tensors in the same queue.

Queue in TensorFlow .NET was originally published in SciSharp STACK on Medium, where people are continuing the conversation by highlighting and responding to this story.

In this article, we’ll implement a simple Convolutional Neural Network model. We’ll implement this model to classify MNIST dataset.

The structure of the neural network we’re going to build is as follows. The hand-written digits images of the MNIST data which has 10 classes (from 0 to 9). The network is with 2 convolutional layers followed by 2 full-connected layers at the end.

neural network architecture

Get started with the implementation:

1. Prepare data

MNIST is dataset of handwritten digits which contains 55,000 examples for training, 5,000 examples for validation and 10,000 example for testing. The digits have been size-normalized and centered in a fixed-size image (28 x 28 pixels) with values from 0 and 1.Each image has been flattened and converted to a 1-D array of 784 features. It’s also kind of benchmark of datasets for deep learning.

We import TensorFlow.NET and NumSharp libraries.

using System;
using NumSharp;
using Tensorflow;
using TensorFlowNET.Examples.Utility;
using static Tensorflow.Python;

const int img_h = 28;
const int img_w = 28;
int n_classes = 10; // Number of classes, one class per digit
int n_channels = 1;

We’ll write the function which automatically loads the MNIST data and returns it in our desired shape and format. There is an MNIST data helper to make life easier.

Datasets mnist;
public void PrepareData()
{
    mnist = MnistDataSet.read_data_sets("mnist", one_hot: true);
}

Other than a function for loading the images and corresponding labels, we still need three more functions:

reformat: reformats the data to the format acceptable for convolutional layer.

private (NDArray, NDArray) Reformat(NDArray x, NDArray y)
{
    var (img_size, num_ch, num_class) = (np.sqrt(x.shape[1]), 1, len(np.unique<int>(np.argmax(y, 1))));
    var dataset = x.reshape(x.shape[0], img_size, img_size, num_ch).astype(np.float32);
    //y[0] = np.arange(num_class) == y[0];
    //var labels = (np.arange(num_class) == y.reshape(y.shape[0], 1, y.shape[1])).astype(np.float32);
    return (dataset, y);
}

randomize: which randomizes the order of images and their labels. At the beginning of each epoch, we will re-randomize the order of data samples to make sure that the trained model is not sensitive to the order of data.

private (NDArray, NDArray) randomize(NDArray x, NDArray y)
{
    var perm = np.random.permutation(y.shape[0]);

np.random.shuffle(perm);
    return (mnist.train.images[perm], mnist.train.labels[perm]);
}

get_next_batch: which only selects a few number of images determined by the batch_size variable (as per Stochastic Gradient Descent method).

private (NDArray, NDArray) get_next_batch(NDArray x, NDArray y, int start, int end)
{
    var x_batch = x[$"{start}:{end}"];
    var y_batch = y[$"{start}:{end}"];
    return (x_batch, y_batch);
}

2. Set Hyperparameters

There’re about 55,000 images in training set, it takes a long time to calculate the gradient of the model using all there images. Therefore we use a small batch of images in each iteration of the optimizer by Stochastic Gradient Descent.

• epoch: one forward pass and one backward pass of all the training examples.
• batch size: the number of training examples in one forward/backward pass. The higher the batch size, the more memory space you’ll need.
• iteration: one forward pass and one backward pass of one batch of images the training examples.

int epochs = 10;
int batch_size = 100;
float learning_rate = 0.001f;
int display_freq = 200; // Frequency of displaying the training results

3. Network configuration

1st convolutional layer:

int filter_size1 = 5;  // Convolution filters are 5 x 5 pixels.
int num_filters1 = 16; //  There are 16 of these filters.
int stride1 = 1;  // The stride of the sliding window

2nd convolutional layer:

int filter_size2 = 5; // Convolution filters are 5 x 5 pixels.
int num_filters2 = 32;// There are 32 of these filters.
int stride2 = 1;  // The stride of the sliding window

Fully-connected layer:

h1 = 128  # Number of neurons in fully-connected layer.

4. Building the neural network

Let’s make some functions to help build computation graph.

variables: We need to define two variables W and b to construct our linear model. We use Tensorflow Variables of proper size and initialization to define them.

// Create a weight variable with appropriate initialization
private RefVariable weight_variable(string name, int[] shape)
{
    var initer = tf.truncated_normal_initializer(stddev: 0.01f);
    return tf.get_variable(name,
                           dtype: tf.float32,
                           shape: shape,
                           initializer: initer);
}

// Create a bias variable with appropriate initialization
private RefVariable bias_variable(string name, int[] shape)
{
    var initial = tf.constant(0f, shape: shape, dtype: tf.float32);
    return tf.get_variable(name,
                           dtype: tf.float32,
                           initializer: initial);
}

2D convolution layer: This layer creates a convolution kernel that is convolved with the layer input to produce a tensor of outputs.

private Tensor conv_layer(Tensor x, int filter_size, int num_filters, int stride, string name)
{
    return with(tf.variable_scope(name), delegate {

var num_in_channel = x.shape[x.NDims - 1];
        var shape = new[] { filter_size, filter_size, num_in_channel, num_filters };
        var W = weight_variable("W", shape);
        // var tf.summary.histogram("weight", W);
        var b = bias_variable("b", new[] { num_filters });
        // tf.summary.histogram("bias", b);
        var layer = tf.nn.conv2d(x, W,
                                 strides: new[] { 1, stride, stride, 1 },
                                 padding: "SAME");
        layer += b;
        return tf.nn.relu(layer);
    });
}

max-pooling layer: Max pooling operation for temporal data.

private Tensor max_pool(Tensor x, int ksize, int stride, string name)
{
    return tf.nn.max_pool(x,
                          ksize: new[] { 1, ksize, ksize, 1 },
                          strides: new[] { 1, stride, stride, 1 },
                          padding: "SAME",
                          name: name);
}

flatten_layer: Flattens the output of the convolutional layer to be fed into fully-connected layer.

private Tensor flatten_layer(Tensor layer)
{
    return with(tf.variable_scope("Flatten_layer"), delegate
                {
                    var layer_shape = layer.TensorShape;
                    var num_features = layer_shape[new Slice(1, 4)].Size;
                    var layer_flat = tf.reshape(layer, new[] { -1, num_features });

return layer_flat;
                });
}

fully-connected layer: Neural network consists of stacks of fully-connected (dense) layers. Having the weight (W) and bias (b) variables, a fully-connected layer is defined as activation(W x X + b). The complete fc_layer function is as below:

private Tensor fc_layer(Tensor x, int num_units, string name, bool use_relu = true)
{
    return with(tf.variable_scope(name), delegate
                {
                    var in_dim = x.shape[1];

var W = weight_variable("W_" + name, shape: new[] { in_dim, num_units });
                    var b = bias_variable("b_" + name, new[] { num_units });

var layer = tf.matmul(x, W) + b;
                    if (use_relu)
                        layer = tf.nn.relu(layer);

return layer;
                });
}

inputs: Now we need to define the proper tensors to feed in the input to our model. Placeholder variable is the suitable choice for the input images and corresponding labels. This allow us to change the inputs (images and labels) to the TensorFlow graph.

with(tf.name_scope("Input"), delegate
     {
         // Placeholders for inputs (x) and outputs(y)
         x = tf.placeholder(tf.float32, shape: (-1, img_h, img_w, n_channels), name: "X");
         y = tf.placeholder(tf.float32, shape: (-1, n_classes), name: "Y");
     });

Placeholder y is the variable for the true labels associated with the images that were input in the placeholder variable x. It holds an arbitrary number of labels and each label is a vector of length num_classes which is 10.

network layers: After creating the proper input, we have to pass it to our model. Since we have a neural network, we can stack multiple fully-connected layers using fc_layer method. Note that we will not use any activation function (use_relu = false) in the last layer. The reason is that we can use tf.nn.softmax_cross_entropy_with_logits to calculate the loss.

var conv1 = conv_layer(x, filter_size1, num_filters1, stride1, name: "conv1");
var pool1 = max_pool(conv1, ksize: 2, stride: 2, name: "pool1");
var conv2 = conv_layer(pool1, filter_size2, num_filters2, stride2, name: "conv2");
var pool2 = max_pool(conv2, ksize: 2, stride: 2, name: "pool2");
var layer_flat = flatten_layer(pool2);
var fc1 = fc_layer(layer_flat, h1, "FC1", use_relu: true);
var output_logits = fc_layer(fc1, n_classes, "OUT", use_relu: false);

loss function, optimizer, accuracy, prediction: After creating the network, we have to calculate the loss and optimize it, we have to calculate the prediction and accuracy.

with(tf.variable_scope("Train"), delegate
     {

with(tf.variable_scope("Optimizer"), delegate
              {
                  optimizer = tf.train.AdamOptimizer(learning_rate: learning_rate, name: "Adam-op").minimize(loss);
              });

with(tf.variable_scope("Accuracy"), delegate
              {
                  var correct_prediction = tf.equal(tf.argmax(output_logits, 1), tf.argmax(y, 1), name: "correct_pred");
                  accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32), name: "accuracy");
              });

with(tf.variable_scope("Prediction"), delegate
              {
                  cls_prediction = tf.argmax(output_logits, axis: 1, name: "predictions");
              });
     });

initialize variables: We have to invoke a variable initializer operation to initialize all variables.

var init = tf.global_variables_initializer();

5. Train

1. After creating the graph, we can train our model. To train the model, we have to create a session and run the graph in the session.

// Number of training iterations in each epoch
var num_tr_iter = y_train.len / batch_size;

var init = tf.global_variables_initializer();
sess.run(init);

float loss_val = 100.0f;
float accuracy_val = 0f;

foreach (var epoch in range(epochs))
{
    print($"Training epoch: {epoch + 1}");
    // Randomly shuffle the training data at the beginning of each epoch 
    (x_train, y_train) = mnist.Randomize(x_train, y_train);

foreach (var iteration in range(num_tr_iter))
    {
        var start = iteration * batch_size;
        var end = (iteration + 1) * batch_size;
        var (x_batch, y_batch) = mnist.GetNextBatch(x_train, y_train, start, end);

// Run optimization op (backprop)
        sess.run(optimizer, new FeedItem(x, x_batch), new FeedItem(y, y_batch));

if (iteration % display_freq == 0)
        {
            // Calculate and display the batch loss and accuracy
            var result = sess.run(new[] { loss, accuracy }, new FeedItem(x, x_batch), new FeedItem(y, y_batch));
            loss_val = result[0];
            accuracy_val = result[1];
            print($"iter {iteration.ToString("000")}: Loss={loss_val.ToString("0.0000")}, Training Accuracy={accuracy_val.ToString("P")}");
        }
    }

// Run validation after every epoch
    var results1 = sess.run(new[] { loss, accuracy }, new FeedItem(x, x_valid), new FeedItem(y, y_valid));
    loss_val = results1[0];
    accuracy_val = results1[1];
    print("---------------------------------------------------------");
    print($"Epoch: {epoch + 1}, validation loss: {loss_val.ToString("0.0000")}, validation accuracy: {accuracy_val.ToString("P")}");
    print("---------------------------------------------------------");
}

6. Test

After the training is done, we have to test our model to see how good it performs on a new dataset.

public void Test(Session sess)
{
    var result = sess.run(new[] { loss, accuracy }, new FeedItem(x, x_test), new FeedItem(y, y_test));
    loss_test = result[0];
    accuracy_test = result[1];
    print("---------------------------------------------------------");
    print($"Test loss: {loss_test.ToString("0.0000")}, test accuracy: {accuracy_test.ToString("P")}");
 print("---------------------------------------------------------");
}

Convolution Neural Network in .NET was originally published in SciSharp STACK on Medium, where people are continuing the conversation by highlighting and responding to this story.

The structure of the neural network we’re going to build is as follows. The hand-written digits images of the MNIST data which has 10 classes (from 0 to 9). The network is with 2 hidden layers: the first layer with 200 hidden units (neurons) and the second one (known as classifier layer) with 10 neurons.

Let’s get started with the implementation step by step:

1. Prepare data

MNIST is dataset of handwritten digits which contains 55,000 examples for training, 5,000 examples for validation and 10,000 example for testing. The digits have been size-normalized and centered in a fixed-size image (28 x 28 pixels) with values from 0 and 1.Each image has been flattened and converted to a 1-D array of 784 features. It’s also kind of benchmark of datasets for deep learning.

We define some variables makes it easier to modify them later. It’s important to note that in a linear model, we have to flatten the input images to a vector.

using System;
using NumSharp;
using Tensorflow;
using TensorFlowNET.Examples.Utility;
using static Tensorflow.Python;

const int img_h = 28;
const int img_w = 28;
int img_size_flat = img_h * img_w; // 784, the total number of pixels
int n_classes = 10; // Number of classes, one class per digit

We’ll write the function which automatically loads the MNIST data and returns it in our desired shape and format. There is an MNIST data helper to make life easier.

Datasets mnist;
public void PrepareData()
{
    mnist = MnistDataSet.read_data_sets("mnist", one_hot: true);
}

Other than a function for loading the images and corresponding labels, we still need two more functions:

randomize: which randomizes the order of images and their labels. At the beginning of each epoch, we will re-randomize the order of data samples to make sure that the trained model is not sensitive to the order of data.

private (NDArray, NDArray) randomize(NDArray x, NDArray y)
{
    var perm = np.random.permutation(y.shape[0]);

    np.random.shuffle(perm);
    return (mnist.train.images[perm], mnist.train.labels[perm]);
}

get_next_batch: which only selects a few number of images determined by the batch_size variable (as per Stochastic Gradient Descent method).

private (NDArray, NDArray) get_next_batch(NDArray x, NDArray y, int start, int end)
{
    var x_batch = x[$"{start}:{end}"];
    var y_batch = y[$"{start}:{end}"];
    return (x_batch, y_batch);
}

2. Set Hyperparameters

There’re about 55,000 images in training set, it takes a long time to calculate the gradient of the model using all there images. Therefore we use a small batch of images in each iteration of the optimizer by Stochastic Gradient Descent.

• epoch: one forward pass and one backward pass of all the training examples.
• batch size: the number of training examples in one forward/backward pass. The higher the batch size, the more memory space you’ll need.
• iteration: one forward pass and one backward pass of one batch of images the training examples.

int epochs = 10;
int batch_size = 100;
float learning_rate = 0.001f;
int h1 = 200; // number of nodes in the 1st hidden layer

3. Building the neural network

Let’s make some functions to help build computation graph.

variables: We need to define two variables W and b to construct our linear model. We use Tensorflow Variables of proper size and initialization to define them.

// weight_variable
var in_dim = x.shape[1];

var initer = tf.truncated_normal_initializer(stddev: 0.01f);
var W = tf.get_variable("W_" + name,
                        dtype: tf.float32,
                        shape: (in_dim, num_units),
                        initializer: initer);

// bias_variable
var initial = tf.constant(0f, num_units);
var b = tf.get_variable("b_" + name,
                        dtype: tf.float32,
                        initializer: initial);

fully-connected layer: Neural network consists of stacks of fully-connected (dense) layers. Having the weight (W) and bias (b) variables, a fully-connected layer is defined as activation(W x X + b). The complete fc_layer function is as below:

private Tensor fc_layer(Tensor x, int num_units, string name, bool use_relu = true)
{
    var in_dim = x.shape[1];

var initer = tf.truncated_normal_initializer(stddev: 0.01f);
    var W = tf.get_variable("W_" + name,
                            dtype: tf.float32,
                            shape: (in_dim, num_units),
                            initializer: initer);

var initial = tf.constant(0f, num_units);
    var b = tf.get_variable("b_" + name,
                            dtype: tf.float32,
                            initializer: initial);

var layer = tf.matmul(x, W) + b;
    if (use_relu)
        layer = tf.nn.relu(layer);

return layer;
}

inputs: Now we need to define the proper tensors to feed in the input to our model. Placeholder variable is the suitable choice for the input images and corresponding labels. This allow us to change the inputs (images and labels) to the TensorFlow graph.

// Placeholders for inputs (x) and outputs(y)
x = tf.placeholder(tf.float32, shape: (-1, img_size_flat), name: "X");
y = tf.placeholder(tf.float32, shape: (-1, n_classes), name: "Y");

Placeholder x is defined for the images, the shape is set to [None, img_size_flat], where None means that the tensor may hold an arbitrary number of images with each image being a vector of length img_size_flat.

Placeholder y is the variable for the true labels associated with the images that were input in the placeholder variable x. It holds an arbitrary number of labels and each label is a vector of length num_classes which is 10.

network layers: After creating the proper input, we have to pass it to our model. Since we have a neural network, we can stack multiple fully-connected layers using fc_layer method. Note that we will not use any activation function (use_relu = false) in the last layer. The reason is that we can use tf.nn.softmax_cross_entropy_with_logits to calculate the loss.

// Create a fully-connected layer with h1 nodes as hidden layer
var fc1 = fc_layer(x, h1, "FC1", use_relu: true);
// Create a fully-connected layer with n_classes nodes as output layer
var output_logits = fc_layer(fc1, n_classes, "OUT", use_relu: false);

loss function: After creating the network, we have to calculate the loss and optimize it, we have to calculate the correct_prediction and accuracy.

// Define the loss function, optimizer, and accuracy
var logits = tf.nn.softmax_cross_entropy_with_logits(labels: y, logits: output_logits);

loss = tf.reduce_mean(logits, name: "loss");

optimizer = tf.train.AdamOptimizer(learning_rate: learning_rate, name: "Adam-op").minimize(loss);

var correct_prediction = tf.equal(tf.argmax(output_logits, 1), tf.argmax(y, 1), name: "correct_pred");

accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32), name: "accuracy");

initialize variables: We have to invoke a variable initializer operation to initialize all variables.

var init = tf.global_variables_initializer();

The complete computation graph is looks like below:

4. Train

After creating the graph, we can train our model. To train the model, we have to create a session and run the graph in the session.

// Number of training iterations in each epoch
var num_tr_iter = mnist.train.labels.len / batch_size;
with(tf.Session(), sess =>
{
    sess.run(init);

float loss_val = 100.0f;
    float accuracy_val = 0f;

foreach (var epoch in range(epochs))
    {
        print($"Training epoch: {epoch + 1}");
        // Randomly shuffle the training data at the beginning of each epoch 
        var (x_train, y_train) = randomize(mnist.train.images, mnist.train.labels);

foreach (var iteration in range(num_tr_iter))
        {
            var start = iteration * batch_size;
            var end = (iteration + 1) * batch_size;
            var (x_batch, y_batch) = get_next_batch(x_train, y_train, start, end);

// Run optimization op (backprop)
            sess.run(optimizer, new FeedItem(x, x_batch), new FeedItem(y, y_batch));

if (iteration % display_freq == 0)
            {
                // Calculate and display the batch loss and accuracy
                var result = sess.run(new[] { loss, accuracy }, new FeedItem(x, x_batch), new FeedItem(y, y_batch));
                loss_val = result[0];
                accuracy_val = result[1];
                print($"iter {iteration.ToString("000")}: Loss={loss_val.ToString("0.0000")}, Training Accuracy={accuracy_val.ToString("P")}");
            }
        }

// Run validation after every epoch
        var results1 = sess.run(new[] { loss, accuracy }, new FeedItem(x, mnist.validation.images), new FeedItem(y, mnist.validation.labels));
        loss_val = results1[0];
        accuracy_val = results1[1];
        print("---------------------------------------------------------");
        print($"Epoch: {epoch + 1}, validation loss: {loss_val.ToString("0.0000")}, validation accuracy: {accuracy_val.ToString("P")}");
        print("---------------------------------------------------------");
    }
});

5. Test

After the training is done, we have to test our model to see how good it performs on a new dataset.

var result = sess.run(new[] { loss, accuracy }, new FeedItem(x, mnist.test.images),  new FeedItem(y, mnist.test.labels));
loss_test = result[0];
accuracy_test = result[1];
print("---------------------------------------------------------");
print($"Test loss: {loss_test.ToString("0.0000")}, test accuracy: {accuracy_test.ToString("P")}");
print("---------------------------------------------------------");

6. Plotting

TODO…

The source code is provided on Github.

Neural Network in TensorFlow.NET was originally published in SciSharp STACK on Medium, where people are continuing the conversation by highlighting and responding to this story.

By running SciSharp’s machine learning libraries on Jupyter Notebook, we can easily use .NET to develop and evaluate machine learning models without the need to install a cumbersome .NET development environment. The C# REPL (Read-Evaluate-Print-Loop) mode allows developers to use CSharp just like a scripting language.

To enable Jupyter to run C# REPL mode, we developed a new Jupyter Kernel ICSharpCore that is responsible for the interaction between the Roslyn compiler and the Jupyter Server through the communication layer provided by ZeroMQ.

In the future we will try to run a complete machine learning model and integrate ploting capabilities.

Updated 05/18/2019

Now we have a complete example for Jupyter in .NET.

Updated 05/24/2019

Added the Plotting feature powered by Plot.NET.

You can download the ipynb file from example folder, or run it through SciSharp Cube docker container in a few minutes.

Running TensorFlow.NET in Jupyter Notebook was originally published in SciSharp STACK on Medium, where people are continuing the conversation by highlighting and responding to this story.

Thanks to the selfless contribution of the SciSharp team, each of your lines of code is full of power.

Updated April 14, 2019

Fixed CondContext, and the unit test are all passed.

Snippet of code output “if x < y then x * 17 else y + 23”

[TestMethod]

public void testCondTrue()

{

  var graph = tf.Graph().as_default();

  with(tf.Session(graph), sess =>

  {

    var x = tf.constant(2, name: "x");

    var y = tf.constant(5, name: "y");

    var z = control_flow_ops.cond(tf.less(x, y),

      () => tf.multiply(x, 17),

      () => tf.add(y, 23));

    int result = z.eval(sess);

    assertEquals(result, 34);

  });

}

The detail can be checked from this issue post.

Conditional Statement Construction of TensorFlow in C# was originally published in SciSharp STACK on Medium, where people are continuing the conversation by highlighting and responding to this story.