cowboyrushforth.com

FLV Duration in Ruby

• Posted: 1 year ago
• Tags:flv, ruby
• 1 comment

There is a lack of decent open source FLV manipulation software out there. There is Yamdi which is a seriously awesome metadata injector, written in C, and then there is FLVTool2, which is a ruby based injector tool. FLVTool2 is really awesome, but it is a serious memory hog and 90% of the files I work with are >1gb size.

I am doing some house cleaning on one project and I have a need to quickly get the duration of an flv video file, but without loading the whole thing into memory, or even reading in the whole file. As an advantage to me, all the files in question have already been injected by yamdi, so the duration is already calculated and inserted into the files. I just need to read it, and be on my merry way.

Here it is:

def duration(filename)
    f = File.open(filename, 'r')
    raise "Could not open File!" unless f

    #check for flv text
    unless f.read(3) === 'FLV'
      raise "This does not seem to be an FLV file."
    end

    #check to make sure we are a video
    f.seek(3)
    is_video = f.read(1).unpack("H*").first.hex
    unless is_video == 1
      raise "This FLV file does not contain video."
    end

    #seek to end, get size
    f.seek(0, IO::SEEK_END)
    length = f.tell

    #calculate tag length
    f.seek(-4, IO::SEEK_END)
    taglen = f.read(4).unpack("H*").first.hex

    #finally get duration
    f.seek(length - taglen, IO::SEEK_SET)
    duration = f.read(3).unpack("H*").first.hex.to_f/1000

    f.close()
    #duration returns in seconds
    duration
  end

I ripped this out of another one of my classes that is already dealing with the flv file in other ways, so there is more validation in place than you see here. This has only been tested with files injected with yamdi 1.3 and 1.4, I have not tested with anything else, but if anyone does please let me know!

If you are interested in doing this in php there is similar scripts here which helped me greatly.

Redmine2Twitter with isgd

• Posted: 1 year ago
• Tags:ruby, twitter
• 0 comments

So as it goes, there were a couple bugs, so I took the opportunity to fix them, and also add some additional features. In this revision we include more of the redmine message that goes along with the activity update as permitted, and then we run the direct link to the redmine page through is.gd for each issue and attach this to the tweet. Additionally I quickly optimized things and made it send is.gd https urls to redmine for my convenience. You can check it out on the second page.
Continue Reading -->

Twitter Command Line

• Posted: 1 year ago
• Tags:ruby, twitter
• 0 comments

I always have a dozen terminals open, so I figured it was time to make a way to post to twitter from the command line. This is extremely simple and can probably be improved, but for now this is working for me:

#!/usr/bin/env ruby
require 'rubygems'
require 'twitter'

if ARGV.size == 0
	puts "\tUsage: tweet i like to eat cheese"
	exit
end

#Load Configuration from ~/.twitter
begin
	config = YAML::load(open(ENV['HOME'] + '/.twitter'))
rescue
	puts "\tConfiguration File Missing"
	puts "\tExample:\n\n"
	puts "\tlogin: mylogin"
	puts "\tpassword: mypassword"
	puts "\n\tput in ~/.twitter\n"
	exit
end

#initialize twitter library
twitter = Twitter::Base.new(config['login'], config['password'])
message = ARGV.join(' ').gsub(/"/, '')
status = twitter.post(message)

puts "\t\tTweet Posted: http://twitter.com/#{status.user.name}/statuses/#{status.id}\n"

Remember to 'gem install twitter' as root, or run as sudo, and also to put your configuration file in ~/.twitter. There is an example in the usage output of the script.

Update: I posted a new and slightly improved version of this for Download here. This version will print out tweets for you if ran with no parameters, and will post any parameters you do use straight to twitter. Also, I found out that the twitter gem I am using already has a decent cli interface. Read about it here.

dspam experiement

• Posted: 1 year ago
• Tags:dspam, ruby
• 0 comments

So finally, SpamAssasin just could not hack it any more on my server. This year there has been a steady influx of spam, and some users of my mail services are literally getting 1 good message out of 100 messages. I feel like I have truly exhausted all SA resources out there. From custom rule sets after more rule sets, after more configuration, SA just can't seem to learn fast enough. I also employ RBL/SBL checks and the works. So, what to do? Enter DSPAM.

Previously, I was a bit weary when I heard about the DSpam project and at the time, SpamAssassin was working for me, so why fix something that is not broken or so I thought. Well that mindset finally passed, and I dove into configuring DSpam for my setup. I run a Postfix/Cyrus type of a virtual email setup that is pretty complicated to say the least. I knew in advance that embarking down this road would mean learning some things and some elbow grease, but that is ok. I finally settled on a configuration that is similar to both Neale's Setup and Cepcep's Setup in different ways. In my setup, I have 2 independent chains, one for inbound email and one for outbound email. I decided to leave my outbound chain alone, and continue to send it thru amavisd. For the inbound chain, all mail is subject to getting sent to dspam, then dspam re-routes it back into postfix with a result attached.

Lastly, I wrote the following script, so that I may simply forward my mails to ham@ or spam@, and have postfix deliver it directly to my script, for retraining purposes.

It is piped to dspam, from postfix, like so:

dspam-retrain   unix    -       n       n       -       10      pipe
  flags=Ru user=dspam argv=ruby /usr/local/bin/dspam-retrain.rb $nexthop $sender $recipient

#!/usr/bin/ruby
# Ruby version of dspam-retrain perl script.
# Perl version: http://dspamwiki.expass.de/DspamRetrainScript
# Author: wsr@rushforthnetworks.com
# License: BSD
# Abstract: setup postfix to pipe mail into this script
#           so that we may then pass it off to dspam in
#           an appropriate way.
#
#           dspam-retrain.rb handles spam-user@domain.tld, and
#                                    spam@domain.tld formats
#
#           If dspam-retrain.rb does not find a signature,
#           it will train dspam using corpus or innoculation
#           mode.      
#
# Requirements: open4 gem is installed
# -----------------------------------------------------------

#### Configuration BEGIN ####################################

#enable logging?

@enable_logging = true
@log_file = '/tmp/dspam_retrain.log'

#what mode to train dspam in if we do not find signature?
#corpus or innoculation

@alternative_mode = 'corpus'

#### Configuration END #####################################


require 'rubygems'
require 'open4'

if @enable_logging
        require 'logger'
        @logfile = File.new(@log_file, 'a+')
        @log = Logger.new(@logfile)
end

def logthis(message)
        if @enable_logging
                @log.info(message)
        end
end

# Get arguments
spam_class  = ARGV[0]
sender = ARGV[1]
recip  = ARGV[2]

logthis("dspam-retrain Started. Arguments: #{spam_class}, #{sender}, #{recip}")

#see if we were passed spam-user@ or just user@
if match = recip.to_s.match(/^(spam|ham)-(\w+)@/)
        user = recip.gsub(/#{match[1]}\-/, '')
elsif match = recip.to_s.match(/^(\w+)@/)
        user = sender
else
        logthis("\tCant't determine user")
        exit 75                
end

signature = String.new
message   = String.new

#loop through email (passed via stdinput)
#search for signature

$stdin.each do |line|
        if line.match(/X-DSPAM-Signature/)
                signature = line.gsub(/X-DSPAM-Signature:/, '')
                #remove any potential whitespace
                signature.strip!
                #since we found signature, break loop
                break
        end
        message << line
end


if signature.length.to_i == 0 
        #we did not find a signature, do normal training

        mode = 'train'

        logthis("\tEmail did not have signature passed in. Attempting #{@alternative_mode} train.")

        #open up dspam with appropriate options
        pid, dspam_in, dspam_out, dspam_err = Open4::popen4 "/usr/bin/dspam --source=#{@alternative_mode} --class=#{spam_class} --user #{user}"

        #attempt to feed message in
        begin
                dspam_in << message
        rescue
                #means dspam closed stdinput because our
                #options failed
                dspam_err.each_line do |o|
                        logthis("\t" + o.gsub(/\n/, ''))
                end
        end
        #close dspams stdinput
        dspam_in.close_write

        #see if dspam left any messages for us
        dspam_out.each_line do |o|
                if o.strip.length == 0 then next end
                logthis("\t" + o.gsub(/\n/, ''))
        end

else
        #we found signature, so we will only pass that.

        mode = 'retrain'

        logthis("\tRetraining Signature: #{signature} for User: #{user} as: #{spam_class}")

        #open up dspam with appropriate options
        pid, dspam_in, dspam_out, dspam_err = Open4::popen4 "/usr/bin/dspam --source=error --signature=#{signature} --class=#{spam_class} --user #{user}"

        #see if dspam left any messages for us
        dspam_out.each_line do |o|
                if o.strip.length == 0 then next end
                logthis("\t" + o.gsub(/\n/,''))
        end
        dspam_err.each_line do |o|
                if o.strip.length == 0 then next end
                logthis("\t" + o.gsub(/\n/,''))
        end
end

ignored, status = Process::waitpid2 pid

#see if we exited cleanly and log it
if status.exitstatus == 0
        logthis("\tMessage successfully #{mode}ed as #{spam_class}")  
else
        logthis("\tMessage NOT #{mode}ed")
end

DRY is pretty cool

• Posted: 2 years ago
• Tags:rails, ruby
• 1 comment

Ok, I have to admit, I was a little turned off by some of the acronyms that were/are being flung around so much when I started getting into rails. Do you like your KISSes DRY? Ugh.

Then it hit me. Both of these things are pretty darn powerful, and can be life changing things. At least life changing if your life is mostly based around writing code. KISS is a funny thing to go around mentioning to a lot of geeks such as myself, but in the end, I really do agree with it and try to fashion code in that style. It stands for Keep-It-Simple-Stupid. Don't complicate things that don't need to be complicated. Enough said.

So this new one, DRY, what's that all about? Well, it stands for Don't-Repeat-Yourself. After working with rails for a bit, I think it has made me a better programmer even in languages and frameworks that are outside of ruby and rails. I often cringe at the thought of having to accept and be cool with new acronyms, but in the end, I don't think that rails and DRY are synchronous, I just think that rails was fashioned with DRY in mind. Today I installed 2 plugins that each took me a days work in one rails application, into a brand new separate rails application. Guess what? It only took me about 2 minutes of tuning to get it to work perfectly. Talk about saving time, its uncanny.

Now I just have to try apply DRY to other projects outside of the rails ones, and try not be upset that they are not rails ;).