I am writing a ChatBot using Svelte and am using the useChat function to interface with my backend to talk to an LLM.
<script lang="ts">
import ChatList from '$lib/components/ChatList.svelte';
import PromptForm from '$lib/components/PromptForm.svelte';
import { useChat, type Message } from 'ai/svelte';
import { API_BASE_URL } from '$lib/config';
export let id: string | undefined;
export let initialMessages: Message[] | undefined;
const { messages, append, isLoading, input } = useChat({
initialMessages,
id,
api: `${API_BASE_URL}/api/chat`,
});
let tempUserInput = '';
async function handleFinished() {
await append({ id, content: tempUserInput, role: 'user' });
tempUserInput = '';
}
</script>
<div class="flex flex-col min-h-screen bg-background">
<div class="flex-1 overflow-y-auto pt-6">
<ChatList {messages} />
</div>
<div class="fixed inset-x-0 bottom-0 bg-metallic-gradient">
<div class="mx-auto sm:max-w-2xl sm:px-4 flex gap-2">
<div
class="space-y-4 border-t bg-background px-4 py-2 shadow-subtle sm:rounded-t-xl sm:border md:py-4 flex gap-2 items-center grow"
>
<PromptForm on:submit={handleFinished} {input} {isLoading} />
</div>
</div>
</div>
</div>
This is working fine for just chats, but sometimes I want to be able to get back images also from the LLM. According to the documentation, I think I can use experimental_attachments
for this.
But I am not sure what to return from my backend to get the information about additional attachments into this experimental_attachments
field.
Right now, I am returning a string, and that automatically seem to go inside the message.content
field.
I tried returning a json
instead that looks like this –
{
'content': self.settings['dev_test_string'],
'role': 'assistant',
'experimental_attachments': {
'content_type': 'img',
'url': 'some_string'
}
}
But this parses the whole of the return json into the message.content
field. What should I return from the backend so that I have more control over what goes into the message object?