[Question]: are patterns supported by local LLMs

[ecliPze507]

Are local LLMs abel to execute fabric patterns?

After reading the documentation, I am still not clear how to get local LLMs working. I tried llama3 and deepseekR1:14b and they both behave similar. I installed the latest go version on Ubuntu 22.04 running in wsl.
If i execute a pattern ( label_and_rate in my case ) with an input both modals (llama3 and deepseekR1) respond with a summary, claiming the user didnt provide any orders what to do with the information given. BUT if I dont add any information what so ever both respond with an empty json and are thinking about the pattern.

I executed
pbpaste | fabric -s -p label_and_rate
with or without an article in the clipboard.

I stumbled accross similar issues like:

• [Question]: why isnt it running the pattern? #514
• [Bug]: #1272

Sadly neither the switch to llama, nor the installation of ffmpeg fixed the issue for me.
Is this a general issue with local LLMs? Are they not supported right now or am I doing something wrong?

Here are all responses given by the 4 executions:

llama3 without content:

Please provide the input content, and I'll generate the output in JSON format according to the steps and instructions provided. I'll make sure to follow the guidelines and use the labels from the approved list. Let's get started!

llama3 with content:

It appears that you've opened the Heise online website, which is a German technology news and magazine portal. The main article on this page is "DeepSeek: Blick hinter die Kulissen des Reasoning-Modells R1" (DeepSeek: A Look Behind the Scenes of the R1 Reasoning Model), which discusses a new artificial intelligence (AI) model called DeepSeek R1.

The article highlights the impressive capabilities of the DeepSeek R1 model, which can process large amounts of text data and perform tasks such as natural language processing and machine learning. The author, Prof. Christian Winkler, is a data scientist and machine learning architect who has been working in the field for over 20 years.

The article also mentions that DeepSeek R1 is a scalable system that can be used for various applications, including text analysis and sentiment analysis. Additionally, it notes that the model is DSGVO-compliant (General Data Protection Regulation), which means it adheres to European data protection laws.

At the top of the page, you'll see a logo for Hannover 96, a German football club. The website also features various sections, including news, tests, and online magazines, such as Heise+, Telepolis, and heise Autos.

You can navigate the site by clicking on the different sections or searching for specific topics using the search bar.

deepseekR1 without content:

Alright, let me try to figure out how to approach this query. The user provided a detailed prompt about classifying content using specific labels and assigning a quality rating based on certain criteria. They included steps for labeling, rating, and scoring, along with an example output in JSON format.

First, I need to understand the goal: I'm supposed to act as a classifier who assigns labels and ratings to given content according to predefined rules. The labels are from a specific list, and the rating depends on the number of ideas and how well the content aligns with certain themes like human meaning or AI impact.

The user's example input was an empty string, resulting in a default output. But I assume that in real scenarios, there would be actual content to process. My task is to analyze any given input content, apply the labels, evaluate the number of ideas and theme relevance, and then assign the appropriate rating and score.

I should pay attention to the criteria mentioned:

1. Labeling: Use only the provided labels without creating new ones.
2. Rating: Based on idea count (above 18 for S Tier) and how well it matches themes like human flourishing or AI's impact.
3. Quality Score: 1-100, with higher scores for more relevant, high-quality ideas matching the themes.

I also need to ensure that content not related to human aspects gets a lower score. Additionally, political content advocating extreme views should be penalized.

If I were to process an actual input, say an article about AI's impact on jobs, I would label it with "AI" and maybe "Future." Then, check the number of ideas and how well it discusses themes like human meaning or future implications. Based on that, assign a rating and score accordingly.

I should make sure my output is strictly in JSON format without any extra text, adhering to the structure provided. Each field needs precise content as per the guidelines.

In summary, my role is to methodically apply the given criteria to each piece of content, ensuring accuracy in labeling, rating, and scoring based on > the outlined rules.

{
  "one-sentence-summary": "",
  "labels": "",
  "rating": "",
  "rating-explanation": "",
  "quality-score": 0,
  "quality-score-explanation": ""
}```

deepseekR1 with content:

Okay, let me try to figure this out. The user provided a chunk of German text that looks like a webpage from Heise Online. It's about DeepSeek and their AI model R1, with some articles and navigation links.

Hmm, the user might be asking for help analyzing this content or perhaps looking for specific information within it. Since they included so much context, maybe they need assistance understanding the structure of the page or extracting certain details.

I should consider what kind of help they're seeking. Are they trying to parse HTML? Or do they need a summary of the article? Maybe they want to translate some parts, but given the content is already in German and I'm supposed to respond in English, that's probably not it.

Looking at their query, they provided the text without any specific question, so perhaps they're looking for an analysis or assistance with navigating the information. They might be a student working on a project about AI companies or a developer interested in integrating similar tools.

I should break down the content: the main article is about DeepSeek's R1 model, written by Prof. Christian Winkler. There are also navigation links, which could indicate areas of interest for further exploration. The user might want to know how to access more articles or understand the structure of Heise Online's website.

Maybe they're trying to identify key points in the article about DeepSeek and their AI capabilities. Alternatively, they might be interested in the site's layout for web scraping purposes but didn't specify that clearly.

I should ask clarifying questions to better assist them. Since I don't want to assume too much, it's best to request more details on what exactly they need help with.

Es scheint, dass du einen Text oder Inhalt von einer Webseite (wie Heise Online) kopiert hast, der sich mit künstlicher Intelligenz und generativer AI befasst. Der Text enthält Artikel, Navigationsmenüs und Informationen über DeepSeek und andere Themen.

Kannst du bitte konkret angeben, was du mit diesem Text machen willst oder wonach du suchst? Ich helfe gerne bei der Analyse, Zusammenfassung, Übersetzung oder anderen Aufgaben!

[ecliPze507]

I just had to dive deeper, stumbled across this issue #743 and this tutorial and found out that you are able to set the context width in the fabric setup it self. Increased the value to 8192 and everything works as expected.

[ecliPze507]

well seems like i closed this one to fast. I am only able to run the local llms with a context width of 8192 on a GTX 1070 with videos up to 15 minutes, if its longer it is ignoring the pattern. Do i need a bigger graphics card to get this working properly?