Scroll Top

Gemma (2b-v2)

C++ (Windows, Linux, MacOS / CUDA and Metal accelerated) port of https://github.com/google/gemma_pytorch.git.

Example Input & Outputs

Inputs Outputs
Which is your favorite Lord of the Rings movie? As an AI, I don’t have personal preferences like a favorite movie.

However, I can tell you that the Lord of the Rings movies are generally considered to be very well-made and beloved by many fans.

Here’s a breakdown of some common opinions:

* **The Fellowship of the Ring:** Many consider this the most faithful adaptation of the book, capturing the epic scope and emotional depth.
* **The Two Towers:** This film is praised for its action sequences and the introduction of new characters like Merry and Pippin.
* **The Return of the King:** This film is often considered the most visually stunning and emotionally resonant, culminating in a satisfying conclusion to the story.

Ultimately, the best Lord of the Rings movie for you depends on your personal preferences.

Demo Code

 1#include "blace_ai.h"
 2#include <fstream>
 3#include <iostream>
 4
 5// include the models you want to use
 6#include "gemma_v2_2b_v2_v1_ALL_export_version_v17.h"
 7
 8int main() {
 9
10  auto gemma = blace::util::registerModel(
11      gemma_v2_2b_v2_v1_ALL_export_version_v17, blace::util::getPathToExe());
12
13  blace::ml_core::InferenceArgsCollection infer_args;
14  infer_args.inference_args.device = blace::util::get_accelerator().value();
15
16  std::vector<std::string> questions = {
17      "What is the answer to life?", "Will ai rule the world?",
18      "Which is your favorite lord of the rings movie?"};
19
20  for (auto str : questions) {
21    auto text_t = CONSTRUCT_OP_GET(blace::ops::FromTextOp(str));
22
23    auto output_len = CONSTRUCT_OP_GET(blace::ops::FromIntOp(200));
24    auto temperature = CONSTRUCT_OP_GET(blace::ops::FromFloatOp(0.));
25    auto top_p = CONSTRUCT_OP_GET(blace::ops::FromFloatOp(0.9));
26    auto top_k = CONSTRUCT_OP_GET(blace::ops::FromIntOp(50));
27
28    // construct inference operation
29    auto infer_op = CONSTRUCT_OP_GET(blace::ops::InferenceOp(
30        gemma_v2_2b_v2_v1_ALL_export_version_v17_IDENT,
31        {text_t, output_len, temperature, top_p, top_k}, infer_args, 0));
32
33    blace::computation_graph::GraphEvaluator evaluator(infer_op);
34    auto answer = evaluator.evaluateToString().value();
35    std::cout << "Answer: " << answer << std::endl;
36
37    // writes text to file
38    std::ofstream out("answer.txt");
39    out << answer;
40    out.close();
41  }
42
43  // unload all models before program exits
44  blace::util::unloadModels();
45
46  return 0;
47}

Follow the 5 minute instructions to build and run the demo.

Tested on version v0.9.28 of blace.ai sdk.

Artifacts

Payload Demo Project Header

License