jody/yacy_search_server
1
0 Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2026-10-25 05:54:15 +00:00
Code Issues Packages Projects Releases Wiki Activity
14,936 Commits 1 Branch 28 Tags
master
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
reger 6b7883900c update bootstrap hosts 2018-07-02 00:00:04 +02:00
reger d5af160e60 upd to slf4j-1.7.25 2018-05-20 21:51:41 +02:00
reger 7525594315 upd to jwat-warc-1.1.1 2018-05-06 00:49:30 +02:00
reger a8234b7ea7 Make sure for image resource url enabled index image pixel size fields are filled 
if at least one of the image size fields is enabled in index (images_height_val,
images_width_val, images_pixel_val). 
Previously all fields were required to be enabled (hint: default setting 
is height + width enabled)
 2018-04-30 04:59:34 +02:00
reger b81debca2e upd to jsoup-1.11.3 2018-04-28 23:24:24 +02:00
reger 508050f79c upd to icu4j-61.1 2018-04-14 16:16:35 +02:00
reger e7971fb888 upd to pdfbox-2.0.9 2018-04-08 20:13:53 +02:00
reger e2b2c89feb upd to jetty-9.4.9.v20180320 2018-04-07 23:39:03 +02:00
reger a57a04a003 upd to commons-codec-1.11 2018-03-19 02:02:35 +01:00
reger aaa0ec6613 upd to commons-compress-1.16.1 2018-02-23 19:17:09 +01:00
reger 5df72c1c65 Remove now obsolete html for language-nav and ISO639 jar reference 2018-02-12 01:16:14 +01:00
reger 87077b8fb6 Adjust and move Language Navigator to be member of the navigatior plugin 
list.
 2018-02-12 00:16:34 +01:00
reger 73c6ce7ae5 upd to httpclient-4.5.5 2018-02-10 20:01:35 +01:00
reger 5aa4fb1144 upd to metadata-extractor-2.11.0.jar 2018-01-27 18:32:45 +01:00
reger cedb53be4e upd to commons-io-2.6 2017-12-28 03:13:42 +01:00
reger f8071ac8ae Make TokenizedStringNavigator (used for keyword search facet) active 
check case insensitive.
As keywords are compared lower case, make sure user input keyword:Key
or keyword:key will be shown as active in facet entry key.
 2017-12-28 02:51:52 +01:00
reger 270b77074e upd to httpclient-4.5.4 and httpmime-4.5.4 2017-12-24 01:34:23 +01:00
reger 6db7f5525b upd to icu4j-60.2 2017-12-24 01:02:18 +01:00
reger c94bc82f6a upd to commons-compress-1.15 2017-12-16 00:49:48 +01:00
reger 20bba135fe Show hide or show public surftip button depending on current config status, 
to show the button to switch the status (hiding button of current status)
 2017-12-10 01:25:20 +01:00
reger cc7a93e6b6 remove deprecated jetty continuation class from urlproxyservlet 
(was a long time carry over, while not supporting async requests)
 2017-12-08 01:01:07 +01:00
reger e5b4799838 upd to Jetty-9.4.8.v20171121 2017-12-07 00:24:33 +01:00
reger 0704b1d644 upd to httpcore-4.4.8 2017-12-04 01:12:50 +01:00
reger a1879115dc upd to Jsoup-1.11.2 2017-11-26 22:01:42 +01:00
reger d5a75537e4 remove redundant setting of timeout for remoteinstance 
and replace depreciated updatesolrclient instantiation with recommended builder
 2017-11-26 02:53:51 +01:00
reger d14c47d4d3 upd to pdfbox-2.0.8.jar 2017-11-05 00:52:14 +01:00
reger b98acb33c3 upd to Solr 6.6.2 2017-10-22 20:00:00 +02:00
reger f8c7d0265e Adjust tags css style in ConfigSearchPage to equal search page 2017-10-07 06:13:22 +02:00
reger c31d94664a Update deprecated SolrInputDocument.addField() with boost value 
remove unused SchemaConfiguration.getDate (as it is designed to return
only past dates which might be unexpected for general configuration schema)
 2017-10-06 20:32:28 +02:00
reger ae1c675c85 fix array out of bounds in YJsonResponseWriter and OpensearchResponsWriter 
on recreation of image url. 
Set parameter of indexList2protocolList to required number of images (image_stubs)
Situation e.g. image_stub(size=15) but images_protocol(size=12)
 2017-10-02 02:51:10 +02:00
reger cbaa492054 upd to Jetty-9.4.7.v20170914 2017-10-02 00:50:30 +02:00
reger ba60f65040 Adjust filetype: query modifier parameter to lower case 
to prevent mismatch on user input with mixed case
Internally file extension are always compared lowercase.
 2017-09-29 00:26:30 +02:00
reger dd82f85953 Add links to the optional keyword tags of search result 
If swichted on link (click) to the tag adds the keyword to the search query.
If a keyword navigator is active the selected keyword adds or replaces 
a query keyword: modifier (currently replace was choosen as multiple 
keywords are not fully supported yet)
 2017-09-28 00:46:49 +02:00
reger c4a7ad2865 update jars for upd solr 6.6. commit for ant 2017-09-17 08:25:14 +02:00
reger b19397bc7c update classpath for Eclipse project config to Solr 6.6.1 2017-09-17 00:29:36 +02:00
reger 2e13ca6e71 update to Solr 6.6.1 
(ant build)
 2017-09-17 00:27:04 +02:00
reger 0cf598659d update to Solr 6.6.1 
(maven build)
 2017-09-17 00:26:18 +02:00
reger 0dbd179479 update maven source and compiler plugin to latest version 2017-09-16 23:58:17 +02:00
reger ba0ba75998 update master.lng, IndexExport_p.html text 2017-09-05 00:51:43 +02:00
reger 31c99b2a8a update master.lng, RankingSolr_p.html text 2017-08-27 04:22:39 +02:00
reger 51a4e03c93 Allow to stop currently running warc import (stop button) 2017-08-20 22:17:27 +02:00
reger f38fb7f02c Add junit test for AbstractOperations.addOperand() 2017-08-14 02:16:43 +02:00
reger 2a07799ad1 Correction of https://github.com/yacy/yacy_search_server/commit/d03e2c98ea6bd5701c8e8257174c439b9c006afb 
Fix Conjunction.addOperator to do nothing if term is empty
prevent to result in query string with repeated logical operator
like "field:term AND AND field:term"
possibliy causing out of mem in postprocessing_doublecontent
 2017-08-14 01:03:15 +02:00
reger d03e2c98ea Fix Conjunction.addOperator to do nothing if term is empty 
prevent to result in query string with repeated logical operator
like "field:term AND AND field:term"
possibliy causing out of mem in postprocessing_doublecontent
 2017-08-14 00:52:03 +02:00
reger b6a41df4f7 Remove deprecated YaCyProxyServlet 
was replaced by UrlProxyServlet
 2017-08-12 21:53:04 +02:00
reger 119b65389d upde to icu4j-59_1.jar 2017-08-10 23:57:37 +02:00
reger 4979439e87 Skip public post of jre version. 
Added to determine switch to java8  https://github.com/yacy/yacy_search_server/commit/596b5dfa5936b25b605c42807730c29a1d08cd15
 2017-08-06 23:41:53 +02:00
reger e918ec199e Replace deprecated ConcurrentHashSet with recommended Java8 
ConcurrentHashMap.newKeySet() in postprocessDocuments()
 2017-08-06 23:26:27 +02:00
reger fb71994342 Harmonizing use of xml reader / sax parser in XMLBlacklistImporter 
eliminating the need for lib/xercesImpl.jar
 2017-08-05 23:47:27 +02:00
reger 275d65fffe Patch last_modified date with internal FirstSeenTime() if no date provided 
to make sure updated documents are indexed with their last-modified
date as provided in current crawl. 
(to patch moddate always with firstseen might bear the risk of miss 
actual updates).
 2017-08-05 22:30:06 +02:00
reger d1b23afed6 Remove obsolete Protocol parameter ttl (time to live) 
not interpreted in target yacy/query.html
also Protocol.querySeed() not used and parameter not interpreted in 
target servlet yacy/query.html
 2017-08-01 00:59:53 +02:00
reger dedc6552d3 upd to poi-3.16.jar 2017-07-31 23:38:10 +02:00
reger 15d78b1064 Replace deprecated getIP with getIPs in Protocol transferURL() and 
getProfile().
Remember used ip for error handling and departInterface
 2017-07-31 01:55:01 +02:00
reger ed36b47bec Replace one more deprecated peerDeparture in Protocol.transferIndex() 
by moving/using interfaceDeparture() in transferRWI()
 2017-07-30 23:02:15 +02:00
reger 37f44941fb upd to pdfbox-2.0.7.jar 2017-07-30 20:09:06 +02:00
reger 41616de0b8 Add SolrConfig ClassicIndexSchemaFactory to prevent Solr startup warning. 
This overrides Solr default to use managed schema. As we don't use
programatic schema changes this directs Solr to use schema.xml, eliminating
the warning.
 2017-07-23 03:55:56 +02:00
reger 44d455dfed upd to jwat-warc-1.1.0.jar 2017-07-16 23:37:28 +02:00
reger 588c6e96fb upd version for typeahead.jquery.js in jslicense.html 2017-07-16 23:35:56 +02:00
reger 3005be7349 Clean up unmaintained and unused AugmentParser trail. 2017-07-15 00:19:23 +02:00
reger e5cff062b5 Clean up redundant but obsolete jquery.rdfquery-core-1.0.js script lib 2017-07-14 23:41:39 +02:00
reger 23bda133d2 Fix css conflict of YMarks.html to make it viewable. 
yacy-ymarks.css sidebar conflicts with bootstraps sidebar (different
overlay settings). Simply renamed it to ymark-sidebar.
 2017-07-09 23:08:54 +02:00
reger af32d291c2 upd to commons-fileupload-1.3.3.jar 2017-07-08 23:46:10 +02:00
reger a21789d4e7 Fix unresolved pattern in api/share.html by init some display var's 2017-07-08 22:46:15 +02:00
reger e6e20dab52 upd to Jetty 9.4.6.v20170531 
Modify loginservice to the changes in Jetty, partially based on pull 
request #101 https://github.com/yacy/yacy_search_server/pull/101 bu @automenta
 2017-07-01 23:58:28 +02:00
reger ddd13b776d Add keyword constraint to rwi query result filter 
To discard rwi results not matching query keyword: parameter
 2017-06-30 02:11:18 +02:00
reger aeeb8a7dd5 upd to jwat-warc-1.0.6.jar 2017-06-25 20:05:37 +02:00
reger f0ba828627 remove unused Solr optional extra handler lib solr-dataimporthandler-6.6.0.jar 2017-06-24 23:15:25 +02:00
reger 1773b61b3e upd to jsoup-1.10.3.jar 2017-06-24 22:54:43 +02:00
reger 83ba45ebae make nsis build script require java 8 2017-06-16 06:31:45 +02:00
reger cf70081cfc update nsi installer java autodl bundleid to use jre-8u131 2017-06-16 02:17:49 +02:00
reger 9220ccbec7 remove reference to velocityresponsewriter in solrconfig.xml 
it is not longer part of solr-core api
http://lucene.apache.org/solr/6_6_0/index.html
 2017-06-16 00:12:09 +02:00
reger 4be4bfbba6 remove sample path setting in solrconfig.xml not valid in Yacy 
resulting in startup stop exception after fresh swithch to 1.921
 2017-06-15 21:02:18 +02:00
reger 510859bcce update maven pom setting to YaCy version 1.921 
java 1.8 and solr 6.6
 2017-06-15 20:24:53 +02:00
reger 632354e2ff Tokenize result entry keywords and add some styling for display 2017-06-04 01:50:40 +02:00
reger c42d17f607 upd to commons-compress-1.14.jar 2017-06-03 21:58:04 +02:00
reger a814f3d885 Introduce keyword query parameter 
This enables keyword navigator to filter on keywords. Added search page
output and layout config for keywords, allowing e.g. in Intranet use
to display the keywords. No styling or links applied to the keyword
text (but is desirable possibly in combination with bootstrap-tagsinput
for future/intranet).
 2017-06-02 01:00:21 +02:00
reger b65a04087b upd to pdfbox-2.0.6.jar 2017-05-24 22:13:42 +02:00
reger 1737af37cf Set request originator to own peer in warc importer 
in addition to change in https://github.com/yacy/yacy_search_server/commit/039162fbf0eca808afd350d360c3bcfe62dc4195
 2017-05-22 01:56:11 +02:00
reger 039162fbf0 Change warc importer to use defaultsurrogate-crawl profile, as reported 
by LA_FORGE http://forum.yacy-websuche.de/viewtopic.php?f=5&t=5990 and
analysed by @luccioman (see comment https://github.com/yacy/yacy_search_server/commit/510f11d3745e14841420781376b733fd248d51f3)
it creates conflict using a other crawlprofile without setting originator.
 2017-05-22 01:34:08 +02:00
reger 077d062be3 Adjust mergeDocuments to keep youngest last-modified date of document 
collection
 2017-05-09 22:52:54 +02:00
reger 9ad4d16829 Add a responsHeader to the solr index export with a format identifier 
and export parameter (in accordance with response xml format) for easier
format detection on import.
 2017-04-30 23:53:52 +02:00
reger 2b03e40134 upd to jwat-1.0.5 2017-04-22 23:32:40 +02:00
reger 7a7da698d4 fix unit test MultiProtocolURL(file) assertion for Windows path with 
drive letter.
 2017-04-20 00:47:52 +02:00
reger c77e43a391 Take out mailto collect in internal parsed document 
As earlier plans to make use of mailto as separate webgraph entity didn't
materialize (see  http://forum.yacy-websuche.de/viewtopic.php?f=8&t=5726&p=32493&hilit=mailto#p32493)
free the unused handling and resources.
 2017-04-20 00:18:18 +02:00
reger bec34d3546 Add url input field as source for WarcImporter 
allowing to import warc from url without prior download.
 2017-04-16 04:25:29 +02:00
reger d3df8a46c4 fix unresolved_pattern on missing post parameter api/message.html 2017-04-14 21:14:26 +02:00
reger 3a71430030 Adjust ConfigSearchPage_p to activated hosts navigator as plugin 2017-04-10 22:58:20 +02:00
reger 7b80189bda Activate hosts navigator plugin. This includes rwi results in the navigator 
count.
This might be tangential related to http://mantis.tokeek.de/view.php?id=736
as the example includes a local index search, while rwi results are not
counted.
 2017-04-10 22:42:06 +02:00
reger 05a1b14b4a add missing text from ConfigRobotsTxt_p to master.lng 
and link to Translation Editor to Translation News page.
 2017-04-09 21:42:05 +02:00
reger a39c00a93f add servlet to list user in UserDB and made user editor available in 
separate servlet for a quick and easy overview of configured user and
selection for edit.
 2017-04-09 02:09:32 +02:00
reger a4498e17c0 fix edit current user form to required post mehtod 
introduced with https://github.com/yacy/yacy_search_server/commit/cde237b68763c542da20038e5f62bea341ae1d37
 2017-04-08 22:54:57 +02:00
reger 46a4aaf09c upd to Solr-5.5.4 2017-04-06 21:18:01 +02:00
reger b7417ac329 Introduce a Keyword search navigator using the index field keywords. 
The keywords field string is split into words as navigator entries.

A keyword navigator facet is essential for search appliance usage were
documents and metadata use often specialized keyword vocabularies to 
filter search results. This navi can be used without custom index schema.

As we don't have defined a search query command to filter "keywords" yet,
the filtering is limited by adding the keyword to the search query.
 2017-04-05 00:08:25 +02:00
reger eddb7a9804 upd to pdfbox-2.0.5.jar and transient dependency xmpcore-5.1.3.jar 
required by metadata-extractor-2.10.1 (fix build.xml compiler warning)
 2017-04-04 00:59:26 +02:00
reger 27884da1ff add CookieTest_p.html text to master.lng 2017-04-03 22:53:07 +02:00
reger c19d60f06b update master.lng with recent text changes 
to IndexExport_p.html, IndexImportWarc_p.html
 2017-04-02 22:30:23 +02:00
reger 9339a6a4c5 use css error class for error msg in IndexImportOAIPMH_p.html, 
adjust to xhtml <p> usage rule
 2017-04-02 20:36:22 +02:00
reger 777cb5b812 remove test case for Standard_MemoryControl which will always fail 
see https://github.com/yacy/yacy_search_server/pull/114
 2017-04-02 03:59:37 +02:00
reger ba339a2a45 Add servlet to import warc file from filesystem IndexImportWarc_p.html. 
Apply Importer interface to WarcImporter
 2017-04-02 03:32:21 +02:00
reger 510f11d374 Implement surrogate import from Warc archives (as first option handle 
warc = Web ARChive File Format.
Warc files with extension .warc or compressed warc.gz can be placed in the
DATA/surrogate/in and contained responses are imported to the index.
The used library is stream based so we can easily extend it later to use
and load warc's from the net.
 2017-03-31 00:58:11 +02:00