Michael Peter Christen
acb77bd68a
implemented (most of)
...
https://github.com/yacy/yacy_search_server/issues/727
some refactorings done as well to support multiple user agents instead
of only one; per request then is one of those user agents selected, not
one random per session.
Also: made sure that no robots.txt is loaded any more in case that the
user agent is not a robot user agent.
2025-08-16 13:33:39 -06:00
Michael Peter Christen
4c953ee8fd
added collection attribute setting for zim and warc imports
...
Having the collection attribute for those imports enables to
use those import methods for YaCy Pack generation which requires
the collection name to produce a pack name.
Fixed also file upload option through browser for both methods.
For large zim/warc files you require large memory settings
to be able to cache the uploaded file.
You also need large memory requirements in your browser to upload
gigabytes of files through multipart/form-data uploads.
2025-08-15 10:26:21 -06:00
Michael Peter Christen
a803c6063b
enhancements to http loading
2025-08-13 00:29:58 -06:00
Michael Peter Christen
88080f48b7
fixes https://github.com/yacy/yacy_search_server/security/dependabot/48
2025-08-12 23:25:48 -06:00
Michael Peter Christen
a4bf2eb2aa
Disabled check for canonical URL
...
This check disables indexing if a canonical tag exists and the current
page is not the canonical page. This check often fails after a
forwarding of a www.-domain to a non-www.-domain where the canonical tag
states that the forwarding page is the canonical page. This makes the
page not indexable. We disable the check therefore by default.
2025-08-12 23:18:28 -06:00
Michael Peter Christen
d1d3f22a27
added 'mix' category for YaCy Packs
2025-08-12 23:08:12 -06:00
Michael Peter Christen
5c4541caee
added Index Pack Manager
2025-08-04 09:39:00 +09:00
Michael Peter Christen
d518dcd883
use minimized exports for packs by default.
2025-08-04 06:11:07 +09:00
Michael Peter Christen
0b89b0e560
added the collection field to minimized exports, needed for packs. Also:
...
replaced sku with urls_s. For this we also relaxed parsing of url fields
in import, also url instead url_s is accepted.
2025-08-04 06:10:23 +09:00
Michael Peter Christen
4beae345f3
more prominent place for collections in crawl start to promote correct
...
use of pack generation using collection names
2025-08-04 06:07:47 +09:00
Michael Peter Christen
02095e5e47
Merge branch 'master' of https://github.com/yacy/yacy_search_server.git
2025-08-01 15:31:15 +09:00
Michael Peter Christen
de8fef9bf1
YaCy Index Pack Generator - refactoring of the SURROGATE process into a
...
"PACK" concept.
2025-08-01 15:30:46 +09:00
Michael Christen
213d8bf3e3
Merge pull request #721 from henschi/fix_dep
...
commons-fileupload 1.6.0 requires commons-io 2.19.0
2025-07-28 20:48:43 +09:00
Sharon Chua
fd6a2bb943
Fix proper encoding in sendDirectory() function
2025-07-28 16:36:42 +08:00
Henschi
a7cc836e64
commons-fileupload 1.6.0 requires commons-io 2.19.0
2025-07-25 17:20:17 +02:00
Michael Peter Christen
055a2eaa25
implemented option for bind address of host.
2025-07-23 21:47:57 +09:00
Michael Peter Christen
6dc8fe872e
implements suggestion from
...
https://github.com/yacy/yacy_search_server/issues/703
2025-07-23 00:03:29 +09:00
Michael Peter Christen
5488bf2e90
fix for https://github.com/yacy/yacy_search_server/issues/717
2025-07-22 19:57:44 +09:00
Michael Peter Christen
7aa11433fd
refactoring and extension of html parser to create markdown
2025-07-22 19:39:11 +09:00
Michael Peter Christen
05f916ef56
less robots tasks by default
2025-07-22 15:53:47 +09:00
Michael Peter Christen
3bb1228e45
fix for NumberFormatException which may have interrupted the parser
2025-07-22 15:53:13 +09:00
Michael Peter Christen
e1390bef69
refactoring + distinguish getMap from getList in table operations
2025-07-22 12:07:35 +09:00
Michael Peter Christen
b030188c27
export less data if not required
2025-07-21 10:53:53 +09:00
Michael Peter Christen
65ea055b58
suppress field "_version_" in jsonl export
2025-07-21 10:07:07 +09:00
Michael Peter Christen
b8875b892f
fix NPE in index import
2025-07-21 09:57:03 +09:00
Michael Peter Christen
281927d505
fix for https://github.com/yacy/yacy_search_server/security/dependabot/47
2025-07-10 11:04:41 +09:00
Michael Christen
7e5a5b04a5
Merge pull request #718 from henschi/nodockeronforks
...
disable docker on forks
2025-06-24 14:50:02 +02:00
Henschi
8829acb697
disable docker build for PRs
2025-06-24 11:47:17 +02:00
Henschi
8a839668cf
disable docker on forks
2025-06-24 11:39:54 +02:00
Michael Peter Christen
86bd58f11f
do not create sha tag images
2025-06-23 06:24:44 +02:00
Michael Peter Christen
e477de9fb8
fix in rag proxy
2025-06-23 06:22:50 +02:00
Michael Peter Christen
f4509ae259
try to fix push problems #4
2025-06-22 16:12:31 +02:00
Michael Peter Christen
5c63775ebc
try to fix push problems #3
2025-06-21 19:36:55 +02:00
Michael Peter Christen
569bb7e2f7
try to fix push problems #2
2025-06-21 19:29:57 +02:00
Michael Peter Christen
2866f913c6
try to fix push problems
2025-06-21 19:28:24 +02:00
Michael Peter Christen
5aaffec11c
introduced reponame for dockerhub
2025-06-21 18:49:07 +02:00
Michael Peter Christen
fa77a20012
enhanced Dockerfiles
2025-06-21 18:16:11 +02:00
Michael Peter Christen
c836eb95a3
fixed another bug in the docker build workflow, #3
2025-06-21 14:40:37 +02:00
Michael Peter Christen
5e9bfc1c70
fixed another bug in the docker build workflow, #2
2025-06-21 14:34:23 +02:00
Michael Peter Christen
ca78e9dea2
fixed another bug in the docker build workflow
2025-06-21 14:24:18 +02:00
Michael Peter Christen
3244fadc9f
remove double suffix
2025-06-21 13:36:00 +02:00
Michael Peter Christen
1563fcd3a8
also produce -ubuntu -tags for default images
2025-06-21 13:14:44 +02:00
Michael Peter Christen
27dbf5a9e5
fix attempt for wrong docker image type (alpine instead ubuntu for latest)
2025-06-21 12:24:51 +02:00
Michael Christen
76ee88c82b
Update build-docker-image.yaml
2025-06-21 00:00:31 +02:00
Michael Christen
205497efdf
disabled selbhosted trigger
2025-06-20 23:52:31 +02:00
Michael Christen
16689362f7
Update build-docker-image.yaml
2025-06-20 23:47:08 +02:00
Michael Christen
13a9329b2d
Update Dockerfile: added retry to prevent failing builds
2025-06-19 18:21:57 +02:00
Michael Christen
09ad31b654
Merge pull request #698 from MisterX2000/master
...
Automate Docker Build Process with Multiarch Support (and GitHub Release action)
2025-06-19 18:11:09 +02:00
Michael Christen
087d924e95
Merge pull request #715 from 3ele-projects/piep/fix_bookmarks_json_api
...
Fix bookmarks json api #713
2025-06-19 11:27:33 +02:00
Michael Christen
02f1d722bb
Merge pull request #704 from jtorelli/add-helm-support
...
Added helm chart
2025-06-19 11:20:01 +02:00
