A lot of edonkey/emule indexing sites are taken down lately.
Would it be possible to a way to keep track of new podcasts or series from emule itself.

My idea is the following.

Instead of going to an indexing site and looking for a new file you are interested in. Have this feature build into emule and make it run offline.

Now a person who is posting checked content on an edonkey site has the problem that this site could be taken down or is dreadfully slow due to high traffic.

If is was possible to post a database like file in the edonkey space that contained all the files that this person has published, people would only have to download this file through emule. If emule could read this file people could select the files they are interested in ad place in into the queue.

If the person wanted to update the database file he published, he would only have to post an update. If the name of this update was formatted correctly emule could be programmed to automatically look for this file and if found download it and add it to the main database file.

The result is some kind of virtual indexing site or subscription. You no longer need to go to a website. Emule would automatically update itself and possibly automatically start downloading the content you are interested in.

This would keep the traffic to the internet site where you found the link to the first database file to a minimum and once a "service" is out there it is impossible to take down and also it is virtually impossible to track the person who posts the updates because there is no website he has to post to.

An example of this is the publication of the content of "The realworld". When this site was taken down a HTML version of its database was published in the emule space and now its content is available offline for everybody and there is no site to take down, because it is already down. If the database of The Realworld could be read by emule and updates where possible. The website would be virtually there and people or organisations with bad intent would not be able to take it down

Sadly I can not program to save my life otherwise I would try my self
Let me know what you think.


Piet

I think we have something like this already - preferences>advanced>auto download. However, don't ask me how to use it as I don't know myself. Hope you can make more sense out of the FAQ than me.

I know about this feature to, but like you have no idea how to get it to work.

But what I am thinking about is beeing able to have a virtual indexing site residing on you own computer and the updates come from the edonky space. Not a website or server that can be taken down.

The automatic download could be a feature of this but the intention is that you could browse something that could look like list you would see in one of the edonkey indexing sites and be able to select the files you want yourself.
You might not want to download everything from a published list.

If you could have a indexing site like shareprovider.com or the old therealworld.de running locally with no more speed problems or organisations trying to take it down, because there is nothing to take down.
And everything updated automatically out of the edonkeyt space.

think about it.

Piet

First you need to convince a forum to release such files.

Without it, the feature is useless.

Also, who garanties that the file is not full of fake links?

Anyone can put a file like that.

If you are thinking of maintaining a local database of files on the ed2k network, that is likely to be impractical. There are several hundred million files in the ed2k network.

But not official releases from oficial groups. I think thats what he meant.

Theres stil the problem i said, thought.

I was not thinking of having an offline index off everything that is on the network but more what Darklord said. A file from the release group.
This way you know you are getting a quality release.

The security is something that was an issue for me too, but it should not be to difficult to encript a database file and only give the creator with the key the right to make addition-files.

If for instance, the LOL group who make all these greate releases, can make an index file with all their releases in there and make sure they are the only once that are able to post additions to this file.
Als these posts can be made more or less anonimously.

You still need somebody or forum to put a link to the initial file up, but there is no more need to revisit the place where you found it once you are "subscribed". Emule would just automatically look for update on the emule servers.


Piet

As muleteer already stated, the feature is called AutoDownload go to the bottom of the following page and read how to use it:
http://emuleplus.info/forum/index.php?show...nQuestions,1257

There you go.

Now you only need to get someone trustworthy to post the file (and lots of bandwidth).

Didnt know about that feature either.

Can see lots of potential (and even something that can work with a thing i created a while back, a irc bot).

Not exactly what you wanted, since it cant be taken from the network, but better than nothing.

Is this a Plus feature only, or there also on other clients (oficial)?

The feature was designed by eMule Plus.

Join the club - haven't been able to figure it out either. Guess I'm not the only dumb guy around here.

The auto download feature is something that could be a feature of what I am thinking off, but you still need a server that is continually hosting the XML files and emule is constantly asking for updates. I was thinking of putting a file in the edonkey space not on a private server. This can still generate a lot of traffic and can be taken down.

An other disadvantage is that you automaticvally starts downloading what is listed in the file. You might just be looking for one episode and not the entire series.

And the main idea here is to have a offline site you could browse and select what you want. Having an auto download beeing part of this would be a nice bonus.

I am not saying that the autodownload feature is not a great idea but my idea is just expaning on it

Piet

You still have the problem of verifying the authenticity of the file.

You need a way to make sure that the file realy comes from the original team.

And no, your solution wont work, since in emule, if a file changes, so does its hash.

Also, having such a database inside Plus would make it really slow and heavy on the memory used.

Yes, you need someone who puts a xml file on a webserver that Plus can download itself.

Brief non-technical explanation of AutoDL feature

Devide people into two groups: Releasers and Downloaders..

- Releaser A creates AutoDL.xml-file (containing one or more files he wants to share to a group of people), and hosts this file on a WebServer (eMule Plus WebServer feature can be used, but any will do, e.g. your ISP etc).
- Releaser B creates AutoDL.xml-file and hosts this file
- Releaser C creates AutoDL.xml-file and hosts this file

- Downloader A wants to download the files Releaser A+C releases, so he specifies both Releaser A's and Releaser C's URLs in AutoDL-section in preferences..
- Downloader B specifies Releaser B's URL in AutoDL
- Downloader C specifies both Releaser B's and Releaser C's URLs in AutoDL

- When Releaser A adds a new file to his XML file, Downloader A automatically get it added to download.
- When Releaser B adds a new file to his XML file, Downloader B+C automatically get it added to download.
- When Releaser C adds a new file to his XML file, Downloader A+C automatically get it added to download.


Hope this makes sense now..

Thanks Fuxie for explaning the auto download feature again. I thought it should work like this when I first heared about it, but was never able to use it because I could not find anybody who was hosting such a file.

The Auto download feature was actually the reason I thought of the offline indexing site.

DARK LORD: Preventing fakes is allways a problem, you are so right. Thats why you have to get the first database file from a trusted source. ( a forum or website you allready know)
Having the excription on the database prevents fakes from beeing added. emule could just check if the update has the right "password". If not, it wont be added.
The first file could be verified by its hashcode. The update cannot work this way because the emule client that is looking for the update cannot know the hash of this unknown file. it can only be identified by its name (for example original file is "index0000.db" and the updates are "index0001.db - index000X.db ") but that is easy to fake. one possibility is to build a verification system into emule that asks the serving emule client that allready has an update to the database file, if the update is the one that is verified to be of the same database the client is asking for.

Every feature you ad will take up memory, but a simple database routine should be not too large. The database itself would just take up harddisk space. It does not need to be loaded completely into memory.
I think adding a feature like this would acutally increase the speed of your emule experience. If you are browsing through an indexing site, it can be very slow, especialy if the site is popular. Having the site on your hdd is lightningly fast and is is updated regularly without the need for somebody to put updates on a website.

Piet

Another possibility is this.

An AutoDonwload using the network, without a fake check, other than the judgement of the users.

I really dont understand what you mean by a password associated with the file; anyway, to check it, you would need to download it, which is what we DONT want (helping propagate false files).
Also didnt understand the last one, but are you saying to trust on the response of the client you are about to download? Also a no-no.


You will have to have at least an index, which wont be that light, and possibly not interesting to everyone.

The problem is the updating however.

As an note, eMule Plus already has a builtin database.

Dark Lord.

If an update is available to the database it needs to be checked to be a valid update. This could be done incorporating some kind of key. Only the creator of the original database has the master key from which updates can be made.

I know its getting kind of technical but there should be a way to only permit valit updates. Someone who is posting a false update could get it downloaded but not installed it the database of the downloader because it lacks the verification key.

It would still be a false download but it would not be propagated because only updates that are verified by your own client should be uploaded again to others by this client.

Adding this feature would ad to the memory footprint of emule but emule is not that big to start out with and allready has the code for database applications in it like you said. I expect it to be a limited increase in memory but modern computers would not notice the difference I think.
It also depends on the size of the database. If it only lists the episodes on one or several series, the database should not exceed 1MB. There are only ed2k links in there with some added text and possibly even thumnails if you want to get creative.
An entire site like the Realworld could get large but even that was like 50MB compessed and that site was getting pretty big.


Piet

Ok, a key. Thats diferent from a password.

Possible, and probably the only way, but not very pratical (how many use validation keys for their email?).


The database would be more than that, since it has to have metadata.

And about the footprint of emule, have you seen how much it consumes running more than a day? Not such a small footprint...

But considering that all the tecnical dificulties are resolved, you still have to convince the release groups to publish such file; that, in my opinion, should be the first step.

Dark Lord: I grand you I have not thought out all the problems that might pop up, allthough discussing it with you was a nice excersize .
The key idea should work I think. This key is supposed to work without the user having to bother with it. Just the creator of the database and the programmer of this feature. It is not needed, but with all the fakes around it would be very wize.


Having a releasegroup updateing a database like this would be ideal, but to start out, anybody could start something like this. All the popular sites started with just one person putting up a webpage and linking files he likes. When other people get hooked you get all the help you need.

But remember it does not have to be a big site linking all the sesies ever made. Everybody who thinks his collection of files, like podcasts, is interesting for somebody else could use this. Is does not require a fast internet connection to serve up thousants of podcast-mp3's or even having a website up 24/7. It is off-line and hosted by everybody in the community who has the same interest.


Piet