active topics restful update

A place to submit all custom code, scripts, and programs.
Forum rules
Do NOT post malicious code or programs. Please review all code posted in this forum before downloading or running any of the code or programs here.

active topics restful update

Post by pretentious on Sat Oct 03, 2015 9:35 am
([msg=89953]see active topics restful update[/msg])

So I decided to work on an unofficial HTS mobile app because https://www.hackthissite.org/forums/viewtopic.php?f=37&t=10674&p=78316 when I say Ima do something, Ima do it. I decided to work on the project iteratively getting little pieces of functionality and piecing them together. Push notifications are a big one that I wanted to implement but just general alerts when something changes are pretty cool. PM's topics posted in, IRC etc.
I developed a basic restful json API thing if you want to call it that. The idea is that the mobile app will store a time stamp when the user exists the app, then when it's reopened, the app will query the web service and get a list of topics that have sprung up since the user logged in. This is pretty sexy in theory and that's why I created it. The wall I ran full speed into is that after some thought, not even much thought, a mobile app would be a major hassle to develop and the pay off would be so little. anyhow, I've cut my losses.

I still feel like this could be useful for anyone wanting to make their own stuff so here it is.
batch_collect.php
Code: Select all
<?php
error_reporting(E_ERROR | E_PARSE);
//header('Content-Type: application/json');
   // don't update this during backups
    $file = 'https://www.hackthissite.org/forums/search.php?search_id=active_topics';
    $html = file_get_contents( $file );
    // initializing dom object
    $dom_object = new DOMDocument();
    $dom_object->loadHtmlFile( $file );
    $xpath = new DOMXPath( $dom_object );
    // last post
    $rows = $xpath->query('//li[@class="row bg1"]//dd[@class="lastpost"]//a/@href');

    // title
    $titles1 = $xpath->query('//li[@class="row bg1"]//dt/a[@class="topictitle"]');
   
    $titles2 = $xpath->query('//li[@class="row bg2"]//dt/a[@class="topictitle"]');

    $tempposts = array();
    foreach ($titles1 as $n){
        $new = array();
        $new['title'] = rtrim($n->textContent);
        //$new['uri'] = "https://www.hackthissite.org/forums/".substr($n->getAttribute("href"),1, strrpos($n->getAttribute("href"), "&sid")-1);
        array_push($tempposts, $new);
    }
        foreach ($titles2 as $n){
        $new = array();
        $new['title'] = $n->textContent;
        //$new['uri'] = "https://www.hackthissite.org/forums".substr($n->getAttribute("href"),1, strrpos($n->getAttribute("href"), "&sid")-1);
        array_push($tempposts, $new);
    }
   $posts = array();
   // ordering posts array
      
   for($i = 0; $i < 12; $i ++){
      array_push($posts, $tempposts[$i]);
      array_push($posts, $tempposts[$i + 12]);
   }
    //print_r($posts);

    //we have our post titles now we get out last posts
    $post = $xpath->query('//li//dd[@class="lastpost"]/span');
   $i = 0;
    foreach ($post as $n ){
        if($n->textContent != 'Last post'){
            $string = ltrim($n->textContent);
            $user = substr($string,3,strpos($string,'   '))."\n";
            $user = rtrim($user);
         $posts[$i]['user'] = $user;
         $date = substr($string,strpos($string,'on ')+3);
         $posts[$i]['date'] = $date;
         
         foreach($n->getElementsByTagName('a') as $k){
            if( strpos($k->getAttribute('href'), 'viewtopic') != FALSE ){
               $posts[$i]['uri'] = "https://www.hackthissite.org/forums". substr($k->getAttribute('href'),1, strpos($k->getAttribute('href'), '&sid')-1);
            }   
         }
      $i ++;
        }
      
    }
$output = fopen("data", "w") ;
    fwrite($output, json_encode($posts));
   fclose($output);
   //echo json_encode($posts);
?>

forum_update.php
Code: Select all
<?php
error_reporting(E_ERROR | E_PARSE);
// out date format which will be sent to the date variable
if( isset($_GET['date']) == true){ // ex Sat Oct 03, 2015 2:19 am
try {
   $date = $_GET['date'];
   $sanitized = preg_replace("/[^a-zA-Z0-9,: ]+/", "", $date);
   $date_object =new DateTime($sanitized);
   header('Content-Type: application/json');
   $raw_data = file_get_contents('data');
   $json = json_decode($raw_data, true);
   //print_r($json);
   //$json =  json_encode($json);
   //echo $raw_data;
   $new_json_array = array();
   foreach($json as $post){
      //print_r($post);
      $temp_date = new DateTime($post['date']);

      if ($date_object < $temp_date){
         array_push($new_json_array, $post);
         //echo $temp_date->format('Y-m-d H:i:s')."<br />";
      }
   }
   echo json_encode($new_json_array);
} catch (Exception $e) {
   //do nothing
}
}
?>

crontab batch_collect to avoid wasting bandwidth and query forum_update
This is seriously the messiest code ever. I do not like PHP
Goatboy wrote:Oh, that's simple. All you need to do is dedicate many years of your life to studying security.

IF you feel like exchanging ASCII arrays, let me know ;)
Can you say brainwashing It's a non stop disco
User avatar
pretentious
Addict
Addict
 
Posts: 1203
Joined: Wed Mar 03, 2010 12:48 am
Blog: View Blog (0)


Re: active topics restful update

Post by tremor77 on Sat Oct 03, 2015 8:43 pm
([msg=89956]see Re: active topics restful update[/msg])

just curious, what sort of things would an HTS mobile app do? forum app would get new topics but would you use it to make replies? not really sure what other useful things the app could have..
Image
User avatar
tremor77
Addict
Addict
 
Posts: 1097
Joined: Wed Mar 31, 2010 12:00 pm
Location: New York
Blog: View Blog (0)


Re: active topics restful update

Post by pretentious on Mon Oct 05, 2015 4:38 am
([msg=89971]see Re: active topics restful update[/msg])

Forum posts/IRC integration was the original idea, Centralizing alerts. When I actually thought about it, yeah I'm not really adding any more functionality and spending a lot of time on it.
Goatboy wrote:Oh, that's simple. All you need to do is dedicate many years of your life to studying security.

IF you feel like exchanging ASCII arrays, let me know ;)
Can you say brainwashing It's a non stop disco
User avatar
pretentious
Addict
Addict
 
Posts: 1203
Joined: Wed Mar 03, 2010 12:48 am
Blog: View Blog (0)


Re: active topics restful update

Post by cyberdrain on Mon Oct 05, 2015 12:16 pm
([msg=89979]see Re: active topics restful update[/msg])

I think it's a good idea, centralised updates about important or less important things can be pretty useful. Besides, HTS has always been more about the community than the missions themselves.
Free your mind / Think clearly
User avatar
cyberdrain
Expert
Expert
 
Posts: 2160
Joined: Sun Nov 27, 2011 1:58 pm
Blog: View Blog (0)


Re: active topics restful update

Post by -Ninjex- on Mon Oct 05, 2015 12:31 pm
([msg=89980]see Re: active topics restful update[/msg])

If you are just requesting and sending data to and from the server, yeah it will be a bit meh. But, there are plenty of things you can do with an application. You don't have to limit yourself to what is here, get creative and think of things that can be of use. Notifications on forum replies, implement a way to have encrypted communication with users directly, maybe a notification when "buddies" come online, etc, etc. I guess what I'm saying is you can take what you have and build on top of that (as you know); you just have to really think of some useful implementations.

Anyways, thanks for sharing the code and maybe someone will pick up on it, or think of some more clever ideas of their own.
image
For those that know
K: 0x2CD8D4F9
User avatar
-Ninjex-
Moderator
Moderator
 
Posts: 1691
Joined: Sun Sep 02, 2012 8:02 pm
Blog: View Blog (0)


Re: active topics restful update

Post by tremor77 on Tue Oct 06, 2015 9:38 pm
([msg=89999]see Re: active topics restful update[/msg])

I wouldn't be opposed to starting up something of an open source project for an HTS mobile app if done in HTML5/Javascript using Cordova/PhoneGap. What would be great about doing a phonegap based HTS app would be is that it's just a webpage, we could just use JQuery UI and AJAX/Restful queries and could be easily developed and tested as a local webpage right on PC, then compiled into an android/ios/windowsphone app via phonegap build tool (iOS requires developer license) but if it turns out nicely, maybe HTS would spring for one. Total client server model no MVC bullshit, if anyone is on board I'm game to lead dev, I just need help with the creativity for "things it should do".
Image
User avatar
tremor77
Addict
Addict
 
Posts: 1097
Joined: Wed Mar 31, 2010 12:00 pm
Location: New York
Blog: View Blog (0)


Re: active topics restful update

Post by pretentious on Thu Oct 08, 2015 6:24 am
([msg=90016]see Re: active topics restful update[/msg])

tremor77 wrote:HTS mobile app if done in HTML5/Javascript using Cordova/PhoneGap. What would be great about doing a phonegap based HTS app would be is that it's just a webpage, we could just use JQuery UI and AJAX/Restful queries and could be easily developed and tested as a local webpage right on PC

What's would the learning curve be like from a Java->Android background? I'm trying to get into the web design world and I'm still wading in the responsive web design pool lol

tremor77 wrote:Total client server model no MVC bullshit

Can you explain your beef with MVC? :P I've bought into it pretty heavy in recent months. separation of model and view makes a huge amount of sense to me.

tremor77 wrote:if anyone is on board I'm game to lead dev, I just need help with the creativity for "things it should do".

It's funny to say in a thread after I've disowned a project but I'd be happy to be a code monkey for this. My problem would be the same as yours though. I don't really see to much that could be created. Ninjex has some good points but I don't think the ends would justify the means. Of course My android dev experience is limited to local storage and basic socket/API stuff. That's sort of why I've given up on this. It took we 2 days just to build a binding API thing in PHP and that's the easy part haha.
On a somewhat unrelated note. How easy would it be to implement something like http://www.businessinsider.com.au/what- ... ing-2014-9 on an android phone? If you're looking for ideas tremor, hintity hint :)
Goatboy wrote:Oh, that's simple. All you need to do is dedicate many years of your life to studying security.

IF you feel like exchanging ASCII arrays, let me know ;)
Can you say brainwashing It's a non stop disco
User avatar
pretentious
Addict
Addict
 
Posts: 1203
Joined: Wed Mar 03, 2010 12:48 am
Blog: View Blog (0)


Re: active topics restful update

Post by tremor77 on Thu Oct 08, 2015 10:32 pm
([msg=90032]see Re: active topics restful update[/msg])

pretentious wrote:
tremor77 wrote:HTS mobile app if done in HTML5/Javascript using Cordova/PhoneGap. What would be great about doing a phonegap based HTS app would be is that it's just a webpage, we could just use JQuery UI and AJAX/Restful queries and could be easily developed and tested as a local webpage right on PC

What's would the learning curve be like from a Java->Android background? I'm trying to get into the web design world and I'm still wading in the responsive web design pool lol


Responsive is all the rage right now lol, but, IMHO it's retarded. Mobile devices can use viewport and pixel ratio meaning, basically you get 1024px width for any webpage regardless of the device. Now you can turn that off if you want things to remain "visibly large" on a mobile device and do the responsive thing.. I guess it's just a matter of style and how you want things to look. Honestly, the learning curve into the world of Javascript - real, meaningful javascript is fucking brutal because it's really freeform so all the onus on maintaining structure is on the developer.

pretentious wrote:
tremor77 wrote:Total client server model no MVC bullshit

Can you explain your beef with MVC? :P I've bought into it pretty heavy in recent months. separation of model and view makes a huge amount of sense to me.


I overstate my dislike, honeslty at the most basic level everything is MVC in some form based on it's definition.. what I'm more sort of opposed to is the use of "templating", and server side manipulation of the actual presentation. (in web apps specifically, not necessarily for 'websites')... I like to develop for the web in a way in which the client (in this case the browser) pretty much gets all the UI and the application logic.. and after that it just sends commands to the server and receives data. I prefer this approach even more when working with HTML5 Canvas :D

pretentious wrote:
tremor77 wrote:if anyone is on board I'm game to lead dev, I just need help with the creativity for "things it should do".
It's funny to say in a thread after I've disowned a project but I'd be happy to be a code monkey for this. My problem would be the same as yours though. I don't really see to much that could be created. Ninjex has some good points but I don't think the ends would justify the means. Of course My android dev experience is limited to local storage and basic socket/API stuff. That's sort of why I've given up on this. It took we 2 days just to build a binding API thing in PHP and that's the easy part haha.


Using phonegap for app development takes out all the the underlying technical stuff, it's literally like.. make pretties, compile and play. Well I make it sound easy there.. but there would be some technical hurdles I think to truly interface with HTS, mainly how to get around same domain origin policies and things like that.. since a phonegap app is basically a webpage hosted on your phone at localhost. There is some chicanery that can be done like using a hidden iframe to actually login to HTS and scarfing it's session.

pretentious wrote:On a somewhat unrelated note. How easy would it be to implement something like http://www.businessinsider.com.au/what- ... ing-2014-9 on an android phone? If you're looking for ideas tremor, hintity hint :)


Which part? There is a lot in that article but I'm assuming the drawing touchy feely stuff? I'd do this extending the phonegap plugin "vibration" and doing like a shared HTML5 canvas via websocket.. that way you could have real-time bidirectional touching and vibration... seems kinda dirty :oops:
Image
User avatar
tremor77
Addict
Addict
 
Posts: 1097
Joined: Wed Mar 31, 2010 12:00 pm
Location: New York
Blog: View Blog (0)


Re: active topics restful update

Post by pretentious on Sun Oct 11, 2015 9:39 pm
([msg=90103]see Re: active topics restful update[/msg])

tremor77 wrote:Responsive is all the rage right now lol, but, IMHO it's retarded. Mobile devices can use viewport and pixel ratio meaning, basically you get 1024px width for any webpage regardless of the device. Now you can turn that off if you want things to remain "visibly large" on a mobile device and do the responsive thing.. I guess it's just a matter of style and how you want things to look.

I can definitely appreciate all that but It's like there is a jet engine in the background while i'm reading it :) I'm definitly more of a back end business logic developer. I developed what I consider to be an amazing website for myself http://stumay.net/ trying to emulate a guitar string. Really proud of it considering my knowledge of js, realize javascript doesn't play nice on mobile platforms and just hack together http://stumay.net/mobile.html fall back page in 2 minutes lol I guess there are some things that people just aren't good at haha
tremor77 wrote:Honestly, the learning curve into the world of Javascript - real, meaningful javascript is fucking brutal because it's really freeform so all the onus on maintaining structure is on the developer.

That's what I feared. Say what you want about those strongly typed languages or whatever they're called, Java C++, they enforce solid programming practices.
There comes a time where you have to just buy in though and I am still determined to get into node.js at some point in my life.

tremor77 wrote:Using phonegap for app development takes out all the the underlying technical stuff, it's literally like.. make pretties, compile and play. Well I make it sound easy there.. but there would be some technical hurdles I think to truly interface with HTS, mainly how to get around same domain origin policies and things like that.. since a phonegap app is basically a webpage hosted on your phone at localhost. There is some chicanery that can be done like using a hidden iframe to actually login to HTS and scarfing it's session.

That's pretty sweet. The whole session handling thing is what made me walk away actually. I can handle a couple of interfaces but I feel like from where I was at, I would be reinventing the wheel on too many levels. I'll talk a look at phonegap :)
tremor77 wrote:
pretentious wrote:On a somewhat unrelated note. How easy would it be to implement something like http://www.businessinsider.com.au/what- ... ing-2014-9 on an android phone? If you're looking for ideas tremor, hintity hint :)


Which part? There is a lot in that article but I'm assuming the drawing touchy feely stuff? I'd do this extending the phonegap plugin "vibration" and doing like a shared HTML5 canvas via websocket.. that way you could have real-time bidirectional touching and vibration... seems kinda dirty :oops:

Yeah that wasn't a great link actually. What I was thinking about was an app that was similar to snapchat drawing but recorded it with timing as well so it could be played back with tweening. Like every 10th of a second sampling to animation. I feel like the tech is definitely there and there might be a market for it and the best bit is I reckon it would take less than 20 hours to develop ;) and I'm a big fan of things that don't take long. Also never discount the trillion dollar sex industry bro :D :twisted:
Goatboy wrote:Oh, that's simple. All you need to do is dedicate many years of your life to studying security.

IF you feel like exchanging ASCII arrays, let me know ;)
Can you say brainwashing It's a non stop disco
User avatar
pretentious
Addict
Addict
 
Posts: 1203
Joined: Wed Mar 03, 2010 12:48 am
Blog: View Blog (0)


Re: active topics restful update

Post by tremor77 on Tue Oct 13, 2015 12:51 am
([msg=90118]see Re: active topics restful update[/msg])

Also never discount the trillion dollar sex industry bro :D :twisted:


I'm thinking a phone app that pairs with a bluetooth "device".. then you connect to a chat session where the other person can control the "device", then you could have hands free sexting.. oh the possibilities. :o
Image
User avatar
tremor77
Addict
Addict
 
Posts: 1097
Joined: Wed Mar 31, 2010 12:00 pm
Location: New York
Blog: View Blog (0)


Next

Return to Custom Code

Who is online

Users browsing this forum: No registered users and 0 guests