jody/yacy_search_server
1
0 Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2026-10-25 05:54:15 +00:00
Code Issues Packages Projects Releases Wiki Activity
14,936 Commits 1 Branch 28 Tags
master
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Michael Peter Christen de973ca444 Merge branch 'master' of https://github.com/yacy/yacy_search_server 2026-09-20 23:44:57 +02:00
Michael Peter Christen 69e9d0d462 added mediawiki/wikipedia to YaCyPack extraction/production and document frequency statistic file generator for advanced ranking 2026-09-20 23:44:16 +02:00
Michael Peter Christen 95078f5f1f Added data structures to store simple csv tables in YaCy. The purpose of 
those tables is to provide a data exchange plattform for agent access
that can communicate with YaCy over a MCP server that provides simple
listTables, readTable and writeTable tool calls.
 2026-09-20 13:24:28 +02:00
Michael Peter Christen 3f21c0c6c1 Bound web structure growth while preserving active crawl roots and connected paths 2026-09-20 12:30:49 +02:00
Michael Peter Christen 42ac9dd630 made the constructor accepting deleteOnClose public, allowing callers to 
choose whether the backing file is deleted when closed
 2026-09-17 19:39:43 +02:00
Michael Peter Christen 8e8f9064bf increased cache time 2026-09-17 19:37:53 +02:00
Michael Peter Christen e171a4a0e5 Keep access tracking cheap during request bursts 2026-09-04 23:31:35 +02:00
Michael Peter Christen c2c7783be8 loading paths now share 256 fixed locks, eliminating per-host lock 
accumulation and the lock-creation race
 2026-09-04 23:23:26 +02:00
Michael Peter Christen d05db9e6bb Introduced immutable indexes to solve the problem of 
dying-peers-when-growing problem:

HeapReader can now open immutable indexes as SSTableHandleMap instances.
Keys and offsets remain file-backed instead of occupying the Java heap.
Lazy working copies avoid additional files until a mutation occurs.
Negative offsets record deletions without tombstone sidecars or RAM
maps. Mutable heaps retain RowHandleMap, while sealed segments save heap
memory.

With immutable indexes the memory consumption is not stopped, it is only
moved from heap memory to system memory. The result should be that there
is no need to assign more memory to YaCy when it gets bigger, it simply
takes it itself from the system memory.

This should also fix
https://github.com/yacy/yacy_search_server/issues/731
 2026-08-30 14:55:29 +02:00
Michael Peter Christen a1be12f5b3 fix for https://github.com/yacy/yacy_search_server/issues/801 2026-08-30 01:13:07 +02:00
Michael Peter Christen 75a0a2be7d fix for https://github.com/yacy/yacy_search_server/issues/804 2026-08-30 00:01:31 +02:00
Michael Peter Christen 3c3a307e8b fix for https://github.com/yacy/yacy_search_server/issues/818 2026-08-28 23:37:51 +02:00
Michael Peter Christen fa8b0e5e32 Fix #815 by downgrading Dropwizard Metrics to 4.2.26 2026-08-21 00:23:08 +02:00
Michael Peter Christen 7bc99cdceb fix for https://github.com/yacy/yacy_search_server/issues/810 2026-08-17 15:00:20 +02:00
Michael Peter Christen b1fd3b3185 Fix Jetty 12 active request accounting 
Track each HTTP request independently and always release its entry on
completion, failure, or async termination. Enforce the request limit
atomically, remove obsolete connection-close accounting, repair stale
cleanup, and persist the configured maximum correctly.

Update UI terminology, help, localization, and focused regression tests.

- Fixes #800
- Fixes #808
 2026-08-16 21:53:13 +02:00
Michael Peter Christen f9c4b97003 Release 1.942 2026-08-15 08:02:33 +02:00
Michael Peter Christen 18e6de8353 removed special handling for qwen3.5 2026-08-15 00:14:31 +02:00
Michael Peter Christen 3df190293a re-enabled gemma4:12b recommendation 2026-08-14 21:24:17 +02:00
Michael Peter Christen b5c9695783 Fix stale config writes during concurrent updates 2026-08-14 21:21:29 +02:00
Michael Peter Christen eff8d81c4e added qwen3.8:27b to recommended model list 2026-08-14 20:52:39 +02:00
Michael Peter Christen 9aa6b93c72 modified model list 2026-08-11 22:59:23 +02:00
Michael Peter Christen bff9cb0562 reservation, rollback, and sequential fallback logic in Heap 2026-08-11 16:31:15 +02:00
Michael Peter Christen 565ab0fd84 Merge branch 'master' of https://github.com/yacy/yacy_search_server 2026-07-30 23:41:09 +02:00
Michael Peter Christen 49d3428a45 Fix error handling for client disconnects 2026-07-30 23:41:05 +02:00
Michael Peter Christen 1f181065ce enhanced Mac App packaging: warn if java is not present or too old 2026-07-14 23:08:27 +02:00
Michael Peter Christen c729aef849 Server-validated X-Real-IP proxy handling, publicPort lifecycle fix, peer-statistic iam parameter, transfer-IP fallback, and matching tests/docs 2026-07-14 22:18:11 +02:00
Michael Peter Christen ece7985d94 proper IP recognition using header X-Real-IP when YaCy is behind a 
reverse proxy
 2026-07-12 17:16:24 +02:00
Michael Peter Christen 92e0b111a0 add solr9-jetty-bridge verification to CI 2026-07-12 13:24:43 +02:00
Michael Peter Christen 302b6af4df Migrated to Solr 9.10.1 and Lucene 9.12.3 2026-07-12 13:13:07 +02:00
Michael Peter Christen d0db6b69c1 Downgraded YaCy to Jetty 12.0.37 2026-07-12 12:43:55 +02:00
Michael Peter Christen 554911a88f Migration to Jetty 12.1.11 2026-07-12 12:03:57 +02:00
Michael Peter Christen b2e9142ea9 towards a Jetty 9 decoupling baseline for the Jetty 12 migration 2026-07-12 00:49:49 +02:00
Michael Peter Christen bfddfe5e7e isolate Solr 9.0 Jetty client dependencies in a relocated bridge 2026-07-11 23:40:15 +02:00
Michael Peter Christen edeb6162c4 de-coupling of security functions from jetty 2026-07-11 20:52:05 +02:00
Michael Peter Christen dd0d4ee962 transform Handler into Servlet-Filter 
to further remove dependencies from jetty
 2026-07-11 13:45:22 +02:00
Michael Peter Christen b32b659023 Added a server interface which is now implemeted by jetty 9 
This will enable a better transition once we migrate to jetty 10
 2026-07-11 12:08:35 +02:00
Michael Peter Christen e42386b3fb removed unnecessary use of jetty 
this is to reduce dependency on jetty before we migrate jetty
 2026-07-11 11:57:09 +02:00
Michael Peter Christen b1380c4a9f cleaned up user authentication 
this removes the userDB-exception in the normal
authentication process which affected mostly wiki
and blog which are not required any more. We also
cleaned up translation messages as this function was
not used at all. This is a required step forward
a renovation of the jetty server.
 2026-07-11 11:11:06 +02:00
Michael Peter Christen f2b0f41d84 migrated jcifs, mime4j to latest 2026-07-10 23:55:32 +02:00
Michael Peter Christen 91a9527bc4 removed org.restlet.jar as it is not needed any more 
restlet was a dependency for Solr 4.8 but not for Solr 9.0 any more.
 2026-07-10 23:45:51 +02:00
Michael Peter Christen f3923a0a0e migrated a large number of libraries to latest 2026-07-10 23:42:16 +02:00
Michael Peter Christen dd0225bdde We migrate to JDK 17 
This  is a hard requirement to be able to migrate
to a more recent jetty version, which may happen sometime soon.
 2026-07-10 22:47:39 +02:00
Michael Peter Christen 3950a4f49b new use case graphics and slightly larger font 2026-07-10 18:08:34 +02:00
Michael Peter Christen dc84868b8b added help files for YaCy UI web pages 2026-07-10 00:20:29 +02:00
Michael Peter Christen ae8941b245 fixed language selector and translations for the selector 2026-07-10 00:19:23 +02:00
Michael Peter Christen 30f66631bc enhanced Network Graph Caching 2026-07-09 22:17:32 +02:00
Michael Peter Christen 55956c502b Complete Chinese, Hindi, Japanese, and Korean translations. 2026-07-09 00:02:50 +02:00
Michael Peter Christen 57d188e69a Major fix for localization: finalizing German, French, Spanish, Italian, 
Greek, Slovak, Ukrainian, Turkish and Russian. Also added full support
for Polish language. All servlets are now fully supported by those
languages.
 2026-07-07 21:36:59 +02:00
Michael Peter Christen 2ca403a1fc fix for spelling in web pages, fixes in locale translations, added 
missing pages in de-locale, removed translation for non-existing pages
 2026-07-07 00:20:00 +02:00
Michael Peter Christen 2e46033b68 better distinct between num_ctx and max_tokens 2026-07-05 21:52:50 +02:00
Michael Peter Christen 0eec5ec6ac another report fix 2026-07-05 13:17:38 +02:00
Michael Peter Christen 9e0a6338cc removed bloat 2026-07-05 12:58:07 +02:00
Michael Peter Christen 6c6148c773 setting max_tokens per model 2026-07-05 12:43:45 +02:00
Michael Peter Christen e6d5aed331 fixed shutdown problem 2026-07-05 12:13:07 +02:00
Michael Peter Christen 35a1d85f8b removed the snapshot feature and all dependencies from non-java code 2026-07-05 12:04:42 +02:00
Michael Peter Christen 9e5958ab0e enhanced logging (to be used in self-enhancement reports) 2026-07-05 09:19:46 +02:00
Michael Peter Christen 28c6e01bcd enhanced download of models, updated model list 2026-07-05 08:48:58 +02:00
Michael Peter Christen e419ceda7c IPv6-aware fixes 
not a full IPv6 support fix, but changes towards
 2026-07-05 08:38:08 +02:00
Michael Peter Christen e5d9efe653 fix for timeout 2026-07-04 17:08:36 +02:00
Michael Peter Christen 8f2f9c3ce4 enhanced gitignore 2026-07-04 14:51:32 +02:00
Michael Peter Christen 51e6c98475 enhanced log reports generation 2026-07-04 14:49:44 +02:00
Michael Peter Christen a72b519440 fix for memory setting 2026-07-04 14:08:46 +02:00
Michael Peter Christen 9f540de067 Enhanced Log Reports 2026-07-04 13:52:47 +02:00
Michael Peter Christen adb04ae397 added llm proxy servlet to augment the RAGProxyServlet with admin 
access. We need this to do remote LLM inference configuration.
 2026-07-04 13:01:38 +02:00
Michael Peter Christen a8b90eb365 Merge branch 'master' of https://github.com/yacy/yacy_search_server.git 2026-07-04 10:57:22 +02:00
Michael Peter Christen e1c988b5c3 added log report generator for recursive self-improvement 2026-06-28 22:49:39 +02:00
Michael Peter Christen 177cc87b89 Merge branch 'master' of https://github.com/yacy/yacy_search_server 2026-06-23 07:47:04 +02:00
Michael Peter Christen 5b6a3fdbc1 Merge branch 'master' of https://github.com/yacy/yacy_search_server 2026-05-31 12:07:41 +02:00
Michael Peter Christen 6d8c08b34a enhanced ai chat: show when tool calls happen and show result of tool 
calls during current chat completion
 2026-05-31 10:22:32 +02:00
Michael Peter Christen d1a32360ab added update_plan tool 2026-05-31 10:21:56 +02:00
Michael Peter Christen 77f556a77d added experimental artifact envelope tool and plotter 2026-05-16 14:21:50 +02:00
Michael Peter Christen 7b303b6f82 Add thinking/tooling/vision/format model capability tests and suppress thinking across LLM calls 2026-03-29 17:33:49 +02:00
Michael Peter Christen f0464e7fbc Release_1.941 - The AI Release 2026-03-29 11:24:35 +02:00
Michael Peter Christen d74e74b809 better ranking for local hits with more context 2026-03-29 11:23:16 +02:00
Michael Peter Christen 5d5e8e889a Restore tool-enabled chat requests from persisted model capabilities 2026-03-29 00:20:49 +01:00
Michael Peter Christen c79b7a19c8 Add configurable maximum length for RAG search documents 2026-03-29 00:19:57 +01:00
Michael Peter Christen a9a365bc75 enhanced tooling/vision testing: persisting all tests 2026-03-28 22:04:33 +01:00
Michael Peter Christen 1f80c6679e Add search tool backed by shared RAG search document output 2026-03-28 21:33:54 +01:00
Michael Peter Christen e0bf6b8c68 Respect persisted tooling capability for chat models 2026-03-28 21:32:37 +01:00
Michael Peter Christen 3c3388201f if webfetch fails, it tells the LLM that this is not fatal and it should go on 2026-03-28 21:29:33 +01:00
Michael Peter Christen 6a8a4cd122 the mermaid generation tool now suggest to call the mermaid visualization tool 2026-03-28 21:29:01 +01:00
Michael Peter Christen 115fdf131b showing different waiting messages for different activities 2026-03-28 17:26:50 +01:00
Michael Peter Christen 106730d36b hero page modification for YaCy Chat 2026-03-28 16:18:42 +01:00
Michael Peter Christen af97d3a682 more models 2026-03-28 16:17:52 +01:00
Michael Peter Christen 31d6de21f4 better tldr handling 2026-03-28 16:17:31 +01:00
Michael Peter Christen 9fff02688c more models 2026-03-28 16:16:08 +01:00
Michael Peter Christen 6f505d1f10 Align RAG retrieval with web search and disable local post-ranking 2026-03-28 11:29:58 +01:00
Michael Peter Christen b6f8b88507 Fix yacychat P2P mode detection for global search visibility 2026-03-28 11:19:37 +01:00
Michael Peter Christen 7b14e31949 Fix for 'n-' fragments that appeared after wrong parsing markdown 2026-03-27 02:34:29 +01:00
Michael Peter Christen 1924f40c01 Enhanced RAG Search: 
- using exactly the same search method as normal search
- removed special search method
 2026-03-27 02:20:25 +01:00
Michael Peter Christen f9933a1f4e Strip markdown syntax from snippet text intended for browser display 2026-03-26 18:45:15 +01:00
Michael Peter Christen 11a5f10c77 modified ranking and query parser to allow more matches and better ranking 2026-03-25 00:13:17 +01:00
Michael Peter Christen 1edee7e127 Merge branch 'master' of https://github.com/yacy/yacy_search_server.git 2026-03-24 00:22:15 +01:00
Michael Peter Christen f4c73aa51a renamed skills -> tools 2026-03-24 00:22:03 +01:00
Michael Peter Christen 4a3b668512 added new models 2026-03-01 17:52:40 +01:00
Michael Peter Christen b8daaacf6a Remove suspicious JavaScript code fragments from snippet lines. 2026-02-17 22:58:02 +01:00
Michael Peter Christen cb3366c3e9 fixed parsing of x-data-elements in <div> 2026-02-17 22:47:32 +01:00
Michael Peter Christen 79c4c4e3dd fixed api-key handling 2026-02-15 21:03:35 +01:00
Michael Peter Christen b8387d4512 added vfs storage to yacychat so the current history does not depend on localStorage any more 2026-02-10 14:57:12 +01:00
Michael Peter Christen 50cf84fd68 added virtual file system that stores users data in the users browser 2026-02-10 14:29:12 +01:00