Gemma (2b-v2)
C++ (Windows, Linux, MacOS / CUDA and Metal accelerated) port of https://github.com/google/gemma_pytorch.git.
Example Input & Outputs
Inputs | Outputs |
Which is your favorite Lord of the Rings movie? | As an AI, I don’t have personal preferences like a favorite movie.
However, I can tell you that the Lord of the Rings movies are generally considered to be very well-made and beloved by many fans. Here’s a breakdown of some common opinions: * **The Fellowship of the Ring:** Many consider this the most faithful adaptation of the book, capturing the epic scope and emotional depth. Ultimately, the best Lord of the Rings movie for you depends on your personal preferences. |
Demo Code
1#include "blace_ai.h"
2#include <fstream>
3#include <iostream>
4
5// include the models you want to use
6#include "gemma_v2_2b_v2_v1_ALL_export_version_v17.h"
7
8int main() {
9
10 auto gemma = blace::util::registerModel(
11 gemma_v2_2b_v2_v1_ALL_export_version_v17, blace::util::getPathToExe());
12
13 blace::ml_core::InferenceArgsCollection infer_args;
14 infer_args.inference_args.device = blace::util::get_accelerator().value();
15
16 std::vector<std::string> questions = {
17 "What is the answer to life?", "Will ai rule the world?",
18 "Which is your favorite lord of the rings movie?"};
19
20 for (auto str : questions) {
21 auto text_t = CONSTRUCT_OP_GET(blace::ops::FromTextOp(str));
22
23 auto output_len = CONSTRUCT_OP_GET(blace::ops::FromIntOp(200));
24 auto temperature = CONSTRUCT_OP_GET(blace::ops::FromFloatOp(0.));
25 auto top_p = CONSTRUCT_OP_GET(blace::ops::FromFloatOp(0.9));
26 auto top_k = CONSTRUCT_OP_GET(blace::ops::FromIntOp(50));
27
28 // construct inference operation
29 auto infer_op = CONSTRUCT_OP_GET(blace::ops::InferenceOp(
30 gemma_v2_2b_v2_v1_ALL_export_version_v17_IDENT,
31 {text_t, output_len, temperature, top_p, top_k}, infer_args, 0));
32
33 blace::computation_graph::GraphEvaluator evaluator(infer_op);
34 auto answer = evaluator.evaluateToString().value();
35 std::cout << "Answer: " << answer << std::endl;
36
37 // writes text to file
38 std::ofstream out("answer.txt");
39 out << answer;
40 out.close();
41 }
42
43 // unload all models before program exits
44 blace::util::unloadModels();
45
46 return 0;
47}
Follow the 5 minute instructions to build and run the demo.
Tested on version v0.9.28 of blace.ai sdk.
Artifacts
Payload | Demo Project | Header |