UP | HOME

HACKING Genshiken

Table of Contents

For those who wish to contribute to Genshiken: Review this file carefully!

Please Note:

Each page, if viewing this document online, is programmed to behave like an info buffer.
Enter `?`, to learn the keybindings for navigating this website. Enter `H`, to return to the main page.

To view the source code for this website, if in GNU IceCat, Enter: `C-c u`.
To view the corresponding source code used to create this website, click the "Show Org source" button at the bottom of this page.

genshiken-logo

Figure 1: HOWTO for HACKING Genshiken

Last modified on December 30, 2016 at 22:15:48

Org mode unicorn logo TEDxGeneva Presentation Emacs

Learn more about this website: View the HTML sources; Validate HTML; JavaScript License Information

Copyleft © 2015-2016 Michael Pagan

Free Documentation, for Free Software This website is licensed under the GNU Free Documentation License. Based on a work at cvs.savannah.gnu.org, with the Invariant Sections being just "What is the purpose of Genshiken?" Code blocks are licensed under the GNU General Public License v3, or later.

1 FAQ

1.1 Q: Why do certain text strings contain strikethroughs in this document?

ANSWER

They either refer to phrases that I don't agree with, or links that don't work in HTML

  • Please note that all strings (with exception to lines speaking of open source software) formatted with a strikethrough, refer to Org targets in Genshiken's source code.
  • Org targets are equivalent to external links, but referencing them online is a bit of a challenge
    • Reason #1 – I have to assume you have the source code already installed
    • Reason #2 – These Org targets were meant to be seen within GNU Emacs, via Org mode

1.2 Q: I'm confused? What does this rainbow of asterisks mean!?

ANSWER

If you are inside of GNU Emacs, then you are in Org mode

  • These asterisks form Outlines
    Ps
    Try reading the rest of that manual while you're at it

1.3 Q: What exactly is the main purpose of Genshiken, anyway?

ANSWER

To free web media!

Virtually all the video you’ve ever watched “on the web” has been funneled through a third-party plugin

– by Mark Pilgrim

Why is it that most companies or website administrators have not started using HTML5, yet? Could it be that they want to hide the locations of their videos via nonfree applications? Do they think that it's ethical to embed videos in a format that restricts users, and encourages them to install proprietary software?

True cyber security/privacy can NOT be maintained, unless free software is utilized! For how can we control our security/privacy with software, if we do not even have control of said software. Free software is the only kind of software that gives us control. I dare you to compare a free software license with a nonfree one, and tell me different! Also, there's no way of telling what malicious functions a plugin or browser-side script may have if it's nonfree!

Videos that require nonfree software, can be loaded via an external player instead; thus, bypassing the need for your browser to behave like a media player via plugins. Genshiken exists so that you no longer have to rely on plugins anymore, especially not the proprietary ones. Most importantly, you are always in control– combined with the 4 Essential Freedoms, Genshiken respects and promotes your freedom!

1.4 Q: What makes Genshiken unique compared to common downloader/streaming clients?

ANSWER

It's a free/libre shell script! Below are some reasons for installing it.

  1. Hackability (i.e. consider the file you're in…)
    • Automatic playlisting
      • Each supported website is treated like a playlist by default
      • You can provide arguments to tweak the playlist
  2. Ease of use
    • Genshiken users don't need to enter a URL to find their videos
    • There's even a Restore Previous Session function
      • In essence, there's no need to remember the commands used to enter a playlist
      • This small outline can't do this particular topic justice; refer to The Manpage
  3. It's a shell-script for crying out loud!
  4. The core of Genshiken is browser and library independent!
    • Why else is my Makefile so small?
  5. Most manuals are tens, hundreds, or even thousands of pages long
    • Genshiken's manual is about 6 pages
    • You CAN master this program!

1.5 Q: Can it really all be done in bash?

ANSWER

Yes! Review how functional Genshiken is already!
link-to-functions

  • The very first program that most users [of a GNU/Linux operating system] encounter, is GNU Bash
    • Before GNU/Linux had a GUI, this was exactly the case. You were dropped into a shell at login
    • Released in 1989, it has since been the default shell for the GNU/Linux operating system
  • A majority of software packages still require the use of shell-scripts for installation
    • Question: What do you think a Makefile is?
      • Answer: A series of shell commands!
  • You have NO reason not to learn it!

1.6 Q: Do I really have to read ALL the Manpages you've listed?

ANSWER

Yes! But if you don't like that answer

Bruce Lee: Absorb what is useful, Discard what is not, Add what is uniquely your own.
Bruce Lee: Be water, my friend

1.7 Q: Bash shell scripts can utilize a variety of programs/languages! Which should I expect?

ANSWER

GNU programs like echo, grep, sed, gawk, etc.

  • Novice level one-liner – Grep and Sed Example
  • Intermediate level one-liner – Find and Xargs Example (That IS a single one-liner)
  • Advanced level one-liner – Sed Example (Ditto…)
  • Expert level one-liner – GAWK Example
  1. See Proficient with a different language

1.8 Q: How should I indent when contributing source code?

ANSWER

Preferably 2 spaces every time you nest within logical blocks of code.

  • Exceptions include:
    • aligning commands after a backquote
    • aligning similar consecutive commands
  • The key is to make the contribution human readable!

1.9 Q: There are too many comments! Do I have to document my code the same way?

ANSWER

Not exactly! It's mainly for reference, and to explain the program flow.

What appears obvious now, may not be in the future– think ahead, before you write! It's also designed for newbies to read it… hint, hint

  1. Specifically, please follow these conventions:
    • Add at least a one-line comment providing a brief synopsis of a function
      • Each new function must be added to the functions directory as a single file
    • Add at least 1-2 lines worth of comments to blocks of code
      • You are encouraged to go in depth for key variables of a function
      • When test conditions seem a bit complicated: Explain the criteria
      • Don't be afraid to fully explain the reasoning behind a single statement
      • Side comments are recommended at specific times– use you're best judgement!
      • Consider adding your 2-cents to the man page, if it takes too much text to explain something

1.10 Q: I want to extend the command-line options of Genshiken. Any advice?

ANSWER

Here's some:

Follow these conventions and ensure your contribution is compatible with this table.

1.11 Q: If I create a new file for Genshiken, is there anything that I should include with it?

ANSWER

Yes! Each new file requires at least 2 things:

A Copyright Notice and a License Notice.
RMS: Without a license notice giving permission to copy and change the file, the file is nonfree.

In essence, you should follow these conventions
  1. Program files should cite the GPL
  2. Documentation should use the GNU Free Documentation License
  3. Small supporting files can use a simple all permissive license
  4. For modules that are treated as a "library", ensure it's compatible with the GPL
Ps
Consider using the overall Copyright Notice and License found in the readme

1.12 Q: Is there anything else I should consider when making a contribution?

ANSWER

Yes! Unlike proprietary software...

1.13 Q: How do I get started?

ANSWER

Simple! Follow the link below.
newbie

1.14 [your questions here!]

2 newbie

2.1 Read the FAQ, before you continue!

2.2 Are you good at bash shell-scripting?

2.2.1 If so, then I could use your help!

  1. E-mail me a link to your free software project so I can have a good idea of your skill level

2.2.2 If not, then that's okay! Begin tackling this newbie outline for success!

2.2.3 Proficient with a different language like Ruby, JavaScript, or Python?

  1. These are the types of languages that operate best on a website
  2. I could use support for languages like those in Genshiken!
    1. Your skills could prove useful for complex operations
    2. For simple operations, regular GNU programs are still preferred
    3. Python developer? See written in Python
    4. JavaScript/Ruby developer? See modifying JavaScript inside HTML tags
    5. Perl/Ruby developer? GNU Awk hacker? See Make REGEX more generic and efficient

2.3 Read Manpages within this HACKING file

2.3.1 Install org-man.el

  1. Relevant information for extending links can be found in Adding hyperlink types
    lynx -dump \
'https://www.gnu.org/software/emacs/manual/html_node/org/Adding-hyperlink-types.html' |\
sed -n '/;;;/,/;;;/ p' > org-man.el

[[ $? -eq 0 ]] &&
echo -e "org-man.el has been downloaded successfully!" ||
echo "org-man.el could not be downloaded!  Check your internet connection" >&2
  2. Move org-man.el to your load path
  3. Finally, load it via your .emacs init file

2.4 TODO Make Genshiken a GNU Project!

2.4.1 Enter C-h g to learn more about the GNU Project

  • If you are not in GNU Emacs: Click here!

2.4.2 If Genshiken becomes a GNU package, then all members of this project will be GNU Hackers!

  1. Please install standards.info & maintain.info, for I shall be referring to them
  2. If these links fail [in Emacs], (standards / maintain): Follow the below steps
    1. Browse The GNU Project Information site
    2. Download the maintain and standards tarballs from GNU's Project Information page
      wget http://www.gnu.org/prep/maintain/maintain.info.tar.gz 2> /dev/null &&
wget http://www.gnu.org/prep/standards/standards.info.tar.gz 2> /dev/null

[[ $? -eq 0 ]] && echo "Successfully downloaded two Project Information tarballs!" ||
                  echo "Could not download the Project Information tarballs!  \
                        Check your internet connections" >&2
    3. Unarchive and uncompress them via tar -xzvf *info.tar.gz
    4. Move the extracted info files to your emacs INFOPATH (e.g. /usr/share/info/emacs-24)
    5. Add the below entries into your dir file:

      GNU organization

      • Maintaining: (maintain) Maintaining GNU software.
      • Standards: (standards) GNU coding standards.

2.5 Grok GNU Bash (I mean it! Don't even continue, if you can't)

2.5.2 While you're at it, grok Abs

2.6 Grok Genshiken

2.6.1 So… Genshiken is under a revision control system? What exactly is CVS?

  1. CVS newbies, read the official documentation: CVS–Concurrent Versions System
    1. Want to get on the fast track?
      1. The project homepage is: http://www.nongnu.org/cvs/
      2. A FAQ
      3. Here's An introduction
      4. Tips for CVS best practices can be found here
  2. CVS Resources on GNU Savannah:
    1. CvsAnonymous - For those who want to access a free software project
      1. Get access to the Genshiken Repository
        1. Genshiken CVS repo
        2. You will use a similar command as this one
        3. Passed the wrong if test? Become a project member!
    2. CvsGettingStarted - For those who want to make a new free software project
      1. Read the free software Project Management-HOWTO
        1. See CVS best practices
        2. See have a free software project
      2. UsingCvs
      3. AccessToCVSROOT
        1. export $CVSROOT in your .shrc file (.e.g. export CVSROOT=/home/user/cvs-projects)
          1. This variable is also used in Genshiken's Makefile
      4. CvsFromBehindFirewall
      5. CvsImportExistingProject
      6. CvsImportExistingRepo
      7. CvsRsyncWriteAccess
      8. CvsDisabling

2.6.2 You are wrong if:

  1. You are not in GNU Emacs!
    1. Quit whichever program you are using right now! This file was designed for Org mode.
    2. For terminal users, enter this: emacs -nw share/doc/genshiken/readme.org
    3. For GUI users, enter this instead: emacs share/doc/genshiken/readme.org
  2. You say:
    1. "Genshiken is open source software!"
    2. "It's more practical than most downloader/streaming clients."
      1. Review Genshiken's License Notice
  3. You read this file before the readme
  4. You are NOT interested in the NEWS
    1. Important announcements will not be sent as a bulk E-mail!
      1. They'll be published as an official news article, so stay in the loop
      2. I will be following these conventions
    2. Comments from a project member can make their way into the NEWS file
      1. Q: Do I expect you to read it?
        1. A: Yes, I do!
        2. Ps: If you are a project member, and I haven't responded to it yet: Reply back.
    3. Subscribe to the Atom feed (e.g. utilize Live Bookmarks or a Feed Reader)
    4. Subscribe to the mailing list. Learn more here.
  5. You haven't downloaded AND installed Genshiken!
  6. You don't have a copy of the repository
    1. See Get access to the Genshiken Repository
  7. You've never executed genshiken –source-code
  8. You haven't tried streaming/downloading with the –verbose option
  9. You've tried tackling the ideas / wishlist or fix bugs while still a newb!
  10. You copied code from another free software project, without giving them proper credit
  11. You've contributed code, but you're mad when I say: it needs to be cleaned up, first.
  12. You've added to The TODO list when it's actually my responsibility as Project Admin
    1. This may change if you become a project member, and I've deemed your skills useful enough
      1. How do I become a project member?
        1. Use the request for inclusion form located on GNU Savannah, to become a member!
        2. What is GNU Savannah?
          1. GNU Savannah is where the gnu roam
          2. This is why you should choose Savannah
          3. Do you have a free software project, too? Register it Here!
        3. If you are a project member, and no longer a newbie: See Genshiken Hacker

3 Genshiken Hacker

3.1 Support an additional website

3.1.1 TODO Pending 5.1

3.1.2 TODO Pending Show-Type configure routine

  1. Will change specific variables that utilize REGEX; make HTML parsing work on all websites
  2. Genshiken may have to be re-structured a bit in order to support more than one website
    1. Let me know if you feel that this is the case!

3.1.3 Study The Main Procedure of Genshiken

3.1.4 Add a new Anime-Type (will soon be "Show-Type")

3.1.5 Review Extract Series List; study how it operates on The Watchcartoononline.COM Network.

  1. This is the first step to take in order to add support to a new website!
  2. You must be able to Parse the HTML via GNU programs like the ones used here

3.1.6 Understand the order of expected execution that genshiken employs

  1. Review the functions directory
    1. These functions are named using numbers that reflect the expected execution order
    2. All functions are sourced at run-time (The Makefile is tasked with modifying this value)
    3. These functions may execute at anytime:
      • func0-cleanup.sh
      • func1-early-termination.sh
      • func2-restore-monitor.sh
    4. All gfunc-* files are executed with:
      • genshiken --get Anime-Type
      • genshiken --stream Anime-Type
    5. All vfunc-* files are executed with:
      • genshiken --view Anime-Type
  2. Study this While Loop.
    1. Each function called here, needs slight modifications in order to support a new website.
    2. The key is: Can it be parsed with free software? See Extract Information for details.

3.1.7 Revise the man page to include the new Show-Type and it's arguments

3.1.8 Those who can accomplish this:

3.1.9 Those who can NOT accomplish this:

3.1.10 Genshiken is already functional! Supporting additional websites will be our main task.

3.2 [more recommendations can be discussed on the mailing list]

3.3 TODO A lot of what happens here will be added to a future info file as a tutorial!

4 ideas / wishlist

4.1 TODO Support more video websites (See Support an additional website)

4.1.1 TODO Add support for Dailymotion

  1. DONE As a third-party link
  2. TODO As a direct link

4.1.2 TODO Add support for Veoh

  1. DONE As a third-party link
  2. TODO As a direct link

4.2 TODO manga/book download procedure

4.3 TODO Support more image websites (See Support an additional website)

4.4 TODO Create more rigorous parsing routines

4.4.1 TODO Learn from youtube-dl - Written in Python, for downloading videos or music

  1. Python developer might be needed here!
  2. Either extend youtube-dl, or add your code directly into Genshiken
  3. If you want, you can even replicate youtube-dl inside of Genshiken
  4. youtube-dl supports 'youtube-like websites'; let's support non-youtube-like websites
    1. This way, we have both ends of the spectrum covered– so to speak

4.4.2 TODO Utilize PhantomJS for analyzing and modifying JavaScript inside HTML tags

  1. JavaScript developer might be needed here!
  2. PhantomJS is only a suggestion! Use what you think will be the best fit for Genshiken.

    If a JavaScript/Ruby developer has a unique solution for dealing with nontrivial nonfree loading scripts: I want to hear it!

4.4.3 TODO Make REGEX more generic and efficient for better use with different websites

  1. Perl or Ruby developer might be needed here!
  2. GNU Awk Hacker might be needed here!
  3. Please refer to change specific variables that utilize REGEX, for your first task

4.5 TODO Add option Episode Guide to the Anime Prompt

4.6 TODO Provide redirection, whenever the current series within a given site can't be loaded

4.6.1 TODO Provide redirection, whenever the current episode within a given series within a given site can't be loaded

4.7 [your hacking ideas here!]

5 fix bugs

5.1 TODO ShowType_Configure (not written, yet): ; not everything is an anime

5.2 TODO Restore_Previous_Session: Different SessionTypes for the same show are ignored (only one is displayed)

5.3 TODO Size_Analysis: Can't retrieve an accurate size of `Dailymotion' videos due to video fragmentation

5.4 TODO Download_File: Allow `Dailymotion' videos to be downloaded, even from a fragmented video playlist

5.5 [found a bug? Report it to the mailing list, and I'll add it here and will find a solution!]

6 ChangeLog discipline

6.1 Based on the GNU coding standards

6.2 Ensure to update the ChangeLog with any changes you make to Genshiken!

  • This is accomplished automatically via cvs2cl (install it if you don't have it)

6.3 The ChangeLog will be for tarball users only (refer to the Makefile for details)

7 Recording Contributors

7.1 Based on the The GNU Maintainers Guide

7.2 Once you start contributing to Genshiken, I will add your name to the AUTHORS file

7.2.1 The format is savannah-user-name:real-name <email-address>:GnuPG-Key-ID:Description of contribution

  • If your contribution does not include code, the first field may be omitted

Don't be shy! I will add your name even if you're contribution consists of:

  • One-line of code
  • A trivial idea you suggested, that makes its way into Genshiken
  • Free documentation that you provide for Genshiken

In essence, any contribution to Genshiken will be recorded in the AUTHORS file. Not sure whether something should be added/not? Discuss it on the mailing list!