Spring AI Alibaba 介紹和功能演示

背景

Spring AI Alibaba 開源項目基于 Spring AI 構(gòu)建，是阿里云通義系列模型及服務(wù)在 Java AI 應(yīng)用開發(fā)領(lǐng)域的最佳實踐，提供高層次的 AI API 抽象與云原生基礎(chǔ)設(shè)施集成方案，幫助開發(fā)者快速構(gòu)建 AI 應(yīng)用。

Spring AI Alibaba 生態(tài)圖如下：

演示

在此節(jié)中，將演示如何使用 Spring AI Alibaba 提供的接口功能完成和 LLMs 的交互。

框架搭建

pom.xml

<dependencies><dependency><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-web</artifactId><version>3.3.4</version></dependency><!-- 最新版本 Spring AI Alibaba --><dependency><groupId>com.alibaba.cloud.ai</groupId><artifactId>spring-ai-alibaba-starter</artifactId><version>1.0.0-M3.1</version></dependency></dependencies><!-- 添加倉庫配置，否則報錯，如果添加之后仍然報錯，刷新 mvn 或者清楚 IDEA 緩存 --><repositories><repository><id>spring-milestones</id><name>Spring Milestones</name><url>https://repo.spring.io/milestone</url><snapshots><enabled>false</enabled></snapshots></repository></repositories><!-- 解決報錯： Name for argument of type [java.lang.String] not specified, and parameter name information not available via reflection. Ensure that the compiler uses the '-parameters' flag --><build><plugins><plugin><groupId>org.apache.maven.plugins</groupId><artifactId>maven-compiler-plugin</artifactId><version>3.11.0</version><configuration><parameters>true</parameters></configuration></plugin></plugins></build></project>

application.yml

spring:ai:dashscope:api-key: ${AI_DASHSCOPE_API_KEY}

啟動類

@SpringBootApplication
public class AIApplication {public static void main(String[] args) {SpringApplication.run(AIApplication.class, args);}}

到此為止，我們已經(jīng)搭建好了一個基本的 AI 應(yīng)用雛形，現(xiàn)在開始和大模型交互 🎉🎉

只演示樸素請求，流式 API 不演示！

Chat 功能

AIChatController.java

@RestController
@RequestMapping("/ai")
public class AIChatController {// 使用高級 Client API，也可以使用低級 ChatModel APIprivate final ChatClient chatClient;public AIChatController(ChatClient.Builder builder) {this.chatClient = builder.build();}@GetMapping("/chat/{prompt}")public String chatWithChatMemory(@PathVariable String prompt) {return chatClient.prompt().user(prompt).call().chatResponse().getResult().getOutput().getContent();}}

如果一切順利，請求 http://localhost:8080/ai/chat/你好接口，將得到以下輸出：

你好！有什么我可以幫助你的嗎？

從代碼中可以看到，使用 Spring AI Alibaba 之后，和模型交互變得非常簡單容易。

但是大模型是無狀態(tài)的，怎么能讓他變得有記憶？Spring AI 提供了 ChatMemory 的接口，只需要調(diào)用接口即可（源碼將在后續(xù)文章中分析）

@RestController
@RequestMapping("/ai")
public class AIChatController {private final ChatClient chatClient;public AIChatController(ChatModel chatModel) {this.chatClient = ChatClient.builder(chatModel).defaultAdvisors(new MessageChatMemoryAdvisor(new InMemoryChatMemory())).build();}@GetMapping("/chat/{chatId}/{prompt}")public String chatWithChatMemory(@PathVariable String chatId,@PathVariable String prompt) {return chatClient.prompt().user(prompt).advisors(a -> a.param(CHAT_MEMORY_CONVERSATION_ID_KEY, chatId).param(CHAT_MEMORY_RETRIEVE_SIZE_KEY, 100)).call().chatResponse().getResult().getOutput().getContent();}}

我們像這樣請求接口：

# 1
input: http://localhost:8080/ai/chat/10001/你好，我是牧生
output：你好，牧生！很高興認(rèn)識你。你可以叫我Qwen，我是阿里云推出的一種超大規(guī)模語言模型。我有強大的語言生成和理解能力，可以進行自然流暢的對話，還能寫故事、寫公文、寫郵件、寫劇本等等，也能表達觀點，玩游戲等。有什么我可以幫助你的嗎？# 2
input：http://localhost:8080/ai/chat/10001/我是誰
output：你剛才提到你的名字是牧生。如果你有任何問題或需要進一步的幫助，隨時告訴我，我很樂意為你服務(wù)！# 當(dāng)切換 chatId 時
input：http://localhost:8080/ai/chat/10000/我叫什么名字
output：您還沒有告訴我您的名字呢。如果您愿意，可以告訴我您希望被稱為什么，或者您想如何介紹自己。

能看到借助 Spring AI 的 ChatMemory 接口，已經(jīng)使大模型變得聰明了起來。 😀

PS：Spring AI Alibaba 的 Chat Memory 功能已經(jīng)在規(guī)劃中了！

Image 功能

本節(jié)我們將展示如何使用 Spring AI Alibaba 提供的 Image API 完成和大模型的圖像交互功能，包含文生圖，多模態(tài)等功能。

AIImageController.java

@RestController
@RequestMapping("/ai")
public class AIImageController {private final ImageModel imageModel;public AIImageController(ImageModel imageModel) {this.imageModel = imageModel;}@GetMapping("/image/{input}")public void image(@PathVariable("input") String input, HttpServletResponse response) {String imageUrl = imageModel.call(new ImagePrompt(input)).getResult().getOutput().getUrl();try {URL url = URI.create(imageUrl).toURL();InputStream in = url.openStream();response.setHeader("Content-Type", MediaType.IMAGE_PNG_VALUE);response.getOutputStream().write(in.readAllBytes());response.getOutputStream().flush();} catch (IOException e) {response.setStatus(HttpServletResponse.SC_INTERNAL_SERVER_ERROR);}}}

請求接口 http://localhost:8080/ai/image/給我一張AI圖片，將會得到以下輸出：

多模態(tài)

MultiModelController.java

多模態(tài)還支持視頻解析和 流式 API。

@RestController
@RequestMapping("/ai")
public class AIImageController {private final ImageModel imageModel;// 加入 chatClientprivate final ChatClient client;public AIImageController(ImageModel imageModel, ChatClient.Builder builder) {this.imageModel = imageModel;this.client = builder.build();}@GetMapping("/image/{input}")public void image(@PathVariable("input") String input, HttpServletResponse response) {ImageOptions options = ImageOptionsBuilder.builder().withModel("wanx-v1").build();String imageUrl = imageModel.call(new ImagePrompt(input, options)).getResult().getOutput().getUrl();try {URL url = URI.create(imageUrl).toURL();InputStream in = url.openStream();response.setHeader("Content-Type", MediaType.IMAGE_PNG_VALUE);response.getOutputStream().write(in.readAllBytes());response.getOutputStream().flush();} catch (IOException e) {response.setStatus(HttpServletResponse.SC_INTERNAL_SERVER_ERROR);}}@GetMapping("/image")public String image(@RequestParam(value = "prompt", required = false, defaultValue = "圖片里是什么")String prompt) throws Exception {// 圖片資源 同時支持讀取本地文件作為輸入List<Media> mediaList = List.of(new Media(MimeTypeUtils.IMAGE_PNG,new URI("https://dashscope.oss-cn-beijing.aliyuncs.com/images/dog_and_girl.jpeg").toURL()));UserMessage message = new UserMessage(prompt, mediaList);message.getMetadata().put(DashScopeChatModel.MESSAGE_FORMAT, MessageFormat.IMAGE);ChatResponse response = client.prompt(new Prompt(message,DashScopeChatOptions.builder().withModel("qwen-vl-max-latest").withMultiModel(true).build())).call().chatResponse();return response.getResult().getOutput().getContent();}}

請求 http://localhost:8080/ai/image 接口，將得到以下輸出：

這張圖片展示了一位女士和一只狗在海灘上互動的溫馨場景。女士坐在沙灘上，面帶微笑，與狗握手。狗戴著項圈，顯得非常溫順和友好。背景是廣闊的海洋和天空，陽光灑在沙灘上，營造出一種溫暖和諧的氛圍。

Audio 功能

本節(jié)我們將展示如何使用 Spring AI Alibaba 提供的 Audio API 完成和大模型的音頻交互功能，包含文生語音，語音轉(zhuǎn)文字等功能。

截止文章發(fā)布期間，Spring AI Alibaba 的語音轉(zhuǎn)錄接口還沒有發(fā)版，有關(guān) stt 和 tts 的更多使用，參考官方 example。

https://github.com/alibaba/spring-ai-alibaba/tree/main/spring-ai-alibaba-examples/audio-example/src/main/java/com/alibaba/cloud/ai/example/audio

AIAudioController.java

@RestController
@RequestMapping("/ai")
public class AIAudioController implements ApplicationRunner {private final SpeechSynthesisModel speechSynthesisModel;private static final String TEXT = "白日依山盡，黃河入海流。";private static final String FILE_PATH = "src/main/resources/gen/tts/";private AIAudioController(SpeechSynthesisModel speechSynthesisModel) {this.speechSynthesisModel = speechSynthesisModel;}@GetMapping("/tts")public void tts() throws IOException {SpeechSynthesisResponse response = speechSynthesisModel.call(new SpeechSynthesisPrompt(TEXT));File file = new File(FILE_PATH + "output.mp3");try (FileOutputStream fos = new FileOutputStream(file)) {ByteBuffer byteBuffer = response.getResult().getOutput().getAudio();fos.write(byteBuffer.array());}catch (IOException e) {throw new IOException(e.getMessage());}}@Overridepublic void run(ApplicationArguments args) {File file = new File(FILE_PATH);if (!file.exists()) {file.mkdirs();}}@PreDestroypublic void destroy() throws IOException {FileUtils.deleteDirectory(new File(FILE_PATH));}}

請求接口，將得到一個語音文件的輸出。

函數(shù)調(diào)用

函數(shù)調(diào)用是為了彌補大模型的訓(xùn)練數(shù)據(jù)落后的問題，用外部的 API 來補充 LLMs 的知識，給用戶最合理的回答。

天氣函數(shù)注冊 MockWeatherService.java

public class MockWeatherService implements Function<MockWeatherService.Request, Response> {@Overridepublic Response apply(Request request) {if (request.city().contains("杭州")) {return new Response(String.format("%s%s晴轉(zhuǎn)多云, 氣溫32攝氏度。", request.date(), request.city()));}else if (request.city().contains("上海")) {return new Response(String.format("%s%s多云轉(zhuǎn)陰, 氣溫31攝氏度。", request.date(), request.city()));}else {return new Response(String.format("暫時無法查詢%s的天氣狀況。", request.city()));}}@JsonInclude(JsonInclude.Include.NON_NULL)@JsonClassDescription("根據(jù)日期和城市查詢天氣")public record Request(@JsonProperty(required = true, value = "city") @JsonPropertyDescription("城市, 比如杭州") String city,@JsonProperty(required = true, value = "date") @JsonPropertyDescription("日期, 比如2024-08-22") String date) {}}

緊接著，我們在 AIChatController 中加入函數(shù)調(diào)用的代碼：

@RestController
@RequestMapping("/ai")
public class AIChatController {private final ChatClient chatClient;public AIChatController(ChatModel chatModel) {this.chatClient = ChatClient.builder(chatModel).defaultAdvisors(new MessageChatMemoryAdvisor(new InMemoryChatMemory())).build();}@GetMapping("/chat/{chatId}/{prompt}")public String chatWithChatMemory(@PathVariable String chatId,@PathVariable String prompt) {return chatClient.prompt().user(prompt).advisors(a -> a.param(CHAT_MEMORY_CONVERSATION_ID_KEY, chatId).param(CHAT_MEMORY_RETRIEVE_SIZE_KEY, 100)).call().chatResponse().getResult().getOutput().getContent();}// 函數(shù)調(diào)用@GetMapping("/weather-service/{city}")public String weatherService(@PathVariable String city) {return chatClient.prompt().function("getWeather", "根據(jù)城市查詢天氣", new MockWeatherService()).user(city).call().content();}}

請求 http://localhost:8080/ai/weather-service/杭州2024年11月29日天氣怎么樣 接口，將得到如下響應(yīng)：

2024年11月29日，杭州的天氣預(yù)報為晴轉(zhuǎn)多云，氣溫為32攝氏度。請根據(jù)天氣情況做好相應(yīng)的準(zhǔn)備。如果您有其他問題，歡迎隨時詢問！

RAG

本節(jié)中我們將使用 ES 作為 RAG 的實現(xiàn)，演示 Spring AI Alibaba RAG 的實現(xiàn)。

在 resource 目錄下準(zhǔn)備一個 system-qa.st

Context information is below.
---------------------
{question_answer_context}
---------------------
Given the context and provided history information and not prior knowledge,
reply to the user comment. If the answer is not in the context, inform
the user that you can't answer the question.

之后準(zhǔn)備一個 df 文件，點擊這里下載：https://github.com/alibaba/spring-ai-alibaba/blob/main/spring-ai-alibaba-examples/rag-example/src/main/resources/data/spring_ai_alibaba_quickstart.pdf

使用 docker compose up -d 啟動一個 es：

準(zhǔn)備配置文件：

config/es.yaml

cluster.name: docker-es
node.name: es-node-1
network.host: 0.0.0.0
network.publish_host: 0.0.0.0
http.port: 9200
http.cors.enabled: true
http.cors.allow-origin: "*"
bootstrap.memory_lock: true# 關(guān)閉認(rèn)證授權(quán) es 8.x 默認(rèn)開啟
# 如果不關(guān)閉，spring boot 連接會 connection closed
xpack.security.enabled: false

docker-compose.yaml

version: '3.3'services:elasticsearch:image: docker.elastic.co/elasticsearch/elasticsearch:8.16.1container_name: elasticsearchprivileged: trueenvironment:- "cluster.name=elasticsearch"- "discovery.type=single-node"- "ES_JAVA_OPTS=-Xms512m -Xmx1096m"- bootstrap.memory_lock=truevolumes:- ./config/es.yaml:/usr/share/elasticsearch/config/elasticsearch.ymlports:- "9200:9200"- "9300:9300"deploy:resources:limits:cpus: "2"memory: 1000Mreservations:memory: 200M

application.yml 中加入 rag 相關(guān)配置：

server:port: 9097spring:ai:dashscope:api-key: ${AI_DASHSCOPE_API_KEY}vectorstore:elasticsearch:index-name: spring-ai-alibaba-indexsimilarity: cosinedimensions: 1536initialize-schema: true

pom.xml 中加入如下配置：

<dependency><groupId>org.springframework.ai</groupId><artifactId>spring-ai-pdf-document-reader</artifactId><version>1.0.0-M3</version>
</dependency><dependency><groupId>org.springframework.ai</groupId><artifactId>spring-ai-elasticsearch-store-spring-boot-starter</artifactId><version>1.0.0-M3</version>
</dependency>

AIRagController.java

package indi.yuluo.controller;import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Objects;import co.elastic.clients.elasticsearch.ElasticsearchClient;
import co.elastic.clients.elasticsearch._types.mapping.DenseVectorProperty;
import co.elastic.clients.elasticsearch._types.mapping.KeywordProperty;
import co.elastic.clients.elasticsearch._types.mapping.ObjectProperty;
import co.elastic.clients.elasticsearch._types.mapping.Property;
import co.elastic.clients.elasticsearch._types.mapping.TextProperty;
import co.elastic.clients.elasticsearch._types.mapping.TypeMapping;
import co.elastic.clients.elasticsearch.indices.CreateIndexResponse;
import co.elastic.clients.elasticsearch.indices.IndexSettings;
import com.alibaba.cloud.ai.advisor.RetrievalRerankAdvisor;
import com.alibaba.cloud.ai.model.RerankModel;
import jakarta.servlet.http.HttpServletResponse;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import reactor.core.publisher.Flux;import org.springframework.ai.autoconfigure.vectorstore.elasticsearch.ElasticsearchVectorStoreProperties;
import org.springframework.ai.chat.client.ChatClient;
import org.springframework.ai.chat.model.ChatModel;
import org.springframework.ai.chat.model.ChatResponse;
import org.springframework.ai.document.Document;
import org.springframework.ai.document.DocumentReader;
import org.springframework.ai.reader.pdf.PagePdfDocumentReader;
import org.springframework.ai.transformer.splitter.TokenTextSplitter;
import org.springframework.ai.vectorstore.SearchRequest;
import org.springframework.ai.vectorstore.VectorStore;
import org.springframework.ai.vectorstore.filter.FilterExpressionBuilder;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.ApplicationArguments;
import org.springframework.boot.ApplicationRunner;
import org.springframework.core.io.Resource;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;/*** @author yuluo* @author <a href="mailto:yuluo08290126@gmail.com">yuluo</a>*/@RestController
@RequestMapping("/ai")
public class AIRagController implements ApplicationRunner {private static final Logger logger = LoggerFactory.getLogger(AIRagController.class);@Value("classpath:/data/spring_ai_alibaba_quickstart.pdf")private Resource PdfResource;@Value("classpath:/prompts/system-qa.st")private Resource systemResource;private static final String textField = "content";private static final String vectorField = "embedding";private final ChatModel chatModel;private final VectorStore vectorStore;private final RerankModel rerankModel;private final ElasticsearchClient elasticsearchClient;private final ElasticsearchVectorStoreProperties options;public AIRagController(ChatModel chatModel,VectorStore vectorStore,RerankModel rerankModel,ElasticsearchClient elasticsearchClient,ElasticsearchVectorStoreProperties options) {this.chatModel = chatModel;this.vectorStore = vectorStore;this.rerankModel = rerankModel;this.elasticsearchClient = elasticsearchClient;this.options = options;}@GetMapping("/rag")public Flux<String> generate(@RequestParam(value = "message", defaultValue = "how to get start with spring ai alibaba?")String message,HttpServletResponse response) throws IOException {// 不設(shè)置返回值會亂碼response.setCharacterEncoding(StandardCharsets.UTF_8.name());return this.retrieve(message).map(x -> x.getResult().getOutput().getContent());}private Flux<ChatResponse> retrieve(String message) throws IOException {// Enable hybrid search, both embedding and full text searchSearchRequest searchRequest = SearchRequest.defaults().withFilterExpression(new FilterExpressionBuilder().eq(textField, message).build());// Step3 - Retrieve and llm generateString promptTemplate = systemResource.getContentAsString(StandardCharsets.UTF_8);;ChatClient chatClient = ChatClient.builder(chatModel).defaultAdvisors(new RetrievalRerankAdvisor(vectorStore,rerankModel,searchRequest,promptTemplate,0.1)).build();return chatClient.prompt().user(message).stream().chatResponse();}@Overridepublic void run(ApplicationArguments args) throws Exception {// 1. parse documentDocumentReader reader = new PagePdfDocumentReader(PdfResource);List<Document> documents = reader.get();logger.info("{} documents loaded", documents.size());// 2. split trunksList<Document> splitDocuments = new TokenTextSplitter().apply(documents);logger.info("{} documents split", splitDocuments.size());// 3. create embedding and store to vector storelogger.info("create embedding and save to vector store");createIndexIfNotExists();vectorStore.add(splitDocuments);}private void createIndexIfNotExists() {try {String indexName = options.getIndexName();Integer dimsLength = options.getDimensions();if (Objects.isNull(indexName) || indexName.isEmpty()) {throw new IllegalArgumentException("Elastic search index name must be provided");}boolean exists = elasticsearchClient.indices().exists(idx -> idx.index(indexName)).value();if (exists) {logger.debug("Index {} already exists. Skipping creation.", indexName);return;}String similarityAlgo = options.getSimilarity().name();IndexSettings indexSettings = IndexSettings.of(settings -> settings.numberOfShards(String.valueOf(1)).numberOfReplicas(String.valueOf(1)));Map<String, Property> properties = new HashMap<>();properties.put(vectorField, Property.of(property -> property.denseVector(DenseVectorProperty.of(dense -> dense.index(true).dims(dimsLength).similarity(similarityAlgo)))));properties.put(textField, Property.of(property -> property.text(TextProperty.of(t -> t))));Map<String, Property> metadata = new HashMap<>();metadata.put("ref_doc_id", Property.of(property -> property.keyword(KeywordProperty.of(k -> k))));properties.put("metadata",Property.of(property -> property.object(ObjectProperty.of(op -> op.properties(metadata)))));CreateIndexResponse indexResponse = elasticsearchClient.indices().create(createIndexBuilder -> createIndexBuilder.index(indexName).settings(indexSettings).mappings(TypeMapping.of(mappings -> mappings.properties(properties))));if (!indexResponse.acknowledged()) {throw new RuntimeException("failed to create index");}logger.info("create elasticsearch index {} successfully", indexName);}catch (IOException e) {logger.error("failed to create index", e);throw new RuntimeException(e);}}}

之后，請求 http://localhost:8080/ai/rag 接口，將得到如下響應(yīng)：

根據(jù)提供的上下文信息，以下是開始使用 Spring AI Alibaba 的步驟： ### 概述 Spring AI Alibaba 實現(xiàn)了與阿里云通義模型的完整適配。下面將介紹如何使用 Spring AI Alibaba 開發(fā)一個基于通義模型服務(wù)的智能聊天應(yīng)用。 ### 快速體驗示例 #### 注意事項 - **JDK 版本**：因為 Spring AI Alibaba 基于 Spring Boot 3.x 開發(fā)，所以本地 JDK 版本要求為 17 及以上。 #### 步驟 1. **下載項目** - 運行以下命令下載源碼，并進入 `helloworld` 示例目錄： ```sh git clone --depth=1 https://github.com/alibaba/spring-ai-alibaba.git cd spring-ai-alibaba/spring-ai-alibaba-examples/helloworld-example ```2. **運行項目** - 首先，需要獲取一個合法的 API-KEY 并設(shè)置 `AI_DASHSCOPE_API_KEY` 環(huán)境變量。你可以跳轉(zhuǎn)到 [阿里云百煉平臺](https://sca.aliyun.com/) 了解如何獲取 API-KEY。 ```sh export AI_DASHSCOPE_API_KEY=${REPLACE-WITH-VALID-API-KEY} ```- 啟動示例應(yīng)用： ```sh ./mvnw compile exec:java -Dexec.mainClass="com.alibaba.cloud.ai.example.helloworld.HelloWorldExample" ```3. **訪問應(yīng)用** - 打開瀏覽器，訪問 `http://localhost:8080/ai/chat?input=給我講一個笑話吧`，向通義模型提問并得到回答。 希望這些步驟能幫助你快速上手 Spring AI Alibaba！如果有任何問題，可以隨時提問。

總結(jié)

Spring AI Alibaba 基于 Spring AI 開發(fā)，并在上層提供更多高級的抽象 API。幫助開發(fā)者構(gòu)建 Java LLMs 應(yīng)用。