Welcome to GOFASTMOTORSPORTS.com ... ... ... Green Flag - Green Flag - Go Go Go ... ... Brought to you by Ricks Satellite - Where the Big Dish Rules!

Ricks Satellite Wildfeed and Backhaul Forum
Register Latest Topics
 
 
 


Reply
  Author   Comment   Page 2 of 21      Prev   1   2   3   4   5   Next   »
midwestmac

Registered:
Posts: 2,275
Reply with quote  #16 
Quote:
Originally Posted by andyinyakima

In updateDVB just Save to File button in the Tuning window after you get Lock. Button is not responsive unless Lock is established. You can change the name of the file if you want before hitting the start button.
Thought you and bluzee were doing that already.

Oh ok, No I was just doing the clean ts dump mostly.

I like your breakdown in post #7  Andy I keep going back to it. I'm still looking at a recording of G18 I made back on 3-15-14 911pm
That GSE header I mentioned  00 01 45 00 05 3C 6C D3 40 00 35 06 5A must be a fragment.
so there's more to it. I haven't put it together yet sure does try to play with VLC though.
I was looking at the "Fragmented PDU chart " EPU1 EPU2 EPU3
from here http://emits.sso.esa.int/emits-doc/REF6_Annex1.pdf
Anyways,
The protocol type 80 ? you weren't sure about I google that "0x80" Could it be "
Service-Specific Connection-Oriented Protocol in a Multilink and Connectionless Environment"
http://en.wikipedia.org/wiki/List_of_IP_protocol_numbers

Added: oops never mind protocol type is 2 bytes, Unless its 0x80 
"Service-Specific Connection-Oriented Protocol in a Multilink and Connectionless Environment" and 0x16 "XEROX NS IDP" (Cisco??)[smile]

__________________
Azbox Ultra, Pansat 2500, Prof7301,Tbs 6925,5980, Genpix 8psk card, Dektec 2137c, Hauppauge 950q
midwestmac

Registered:
Posts: 2,275
Reply with quote  #17 
Quote:
Originally Posted by andyinyakima
midwestmac,

I found the code and down loaded it.

I think it is aimed at 6 byte label and two way simulation.
I read in the changelog:

29 May 2012 - release 1.0.1
  first public release of GSE library
  some standard secifications are not implemented
    - no "3-byte label", "label re-use" and "no label" support
    - no extension support


I am trying to grasp the deencap.c code, that and some of the files in common.

Time will tell if it is useful. The guys who wrote it might have been there at the start of GSE.

The more one looks at it the more a person might understand all of GSE capabilities.



Did you ever see code about "BBheader's" in this one I posted? all I see is "GSE". If so Where? I was just browsing it. I didn't download. Or anybody who knows where they might have put it, I don't Thanks.
https://launchpad.net/libgse/

ADDED: Never mind Found it 

And thanks for this one. I found what I need there 
https://bugs.wireshark.org/bugzilla/attachment.cgi?id=8834

__________________
Azbox Ultra, Pansat 2500, Prof7301,Tbs 6925,5980, Genpix 8psk card, Dektec 2137c, Hauppauge 950q
midwestmac

Registered:
Posts: 2,275
Reply with quote  #18 
Andy did you get that recording of g18 I sent I guess? I was looking at it some more and it appears that maybe more BBheaders
Besides the BBheader starting with "42" I found 
39 00 02 00 80
49 00 02 00 80

In binary
39 = 00111001 -generic packetized, SIS, CCM, ISSYI(not active), Null Packet deletion (not active), roll off 0.25
49 = 01001001 -generic continuous, MIS, ACM, ISSYI(not active) Null packet deletion (not active), roll off 0.25

If thats the case? WOW what a mess [smile] Thought I'd share if you wanted to look at it




__________________
Azbox Ultra, Pansat 2500, Prof7301,Tbs 6925,5980, Genpix 8psk card, Dektec 2137c, Hauppauge 950q
andyinyakima

Avatar / Picture

Registered:
Posts: 1,520
Reply with quote  #19 
Quote:
Originally Posted by midwestmac
Andy did you get that recording of g18 I sent I guess? I was looking at it some more and it appears that maybe more BBheaders
Besides the BBheader starting with "42" I found 
39 00 02 00 80
49 00 02 00 80

In binary
39 = 00111001 -generic packetized, SIS, CCM, ISSYI(not active), Null Packet deletion (not active), roll off 0.25
49 = 01001001 -generic continuous, MIS, ACM, ISSYI(not active) Null packet deletion (not active), roll off 0.25

If thats the case? WOW what a mess [smile] Thought I'd share if you wanted to look at it





mac

we might have to apply the crc-8 to authenticate the BBheaders. For a while I think I will just concentrate on 42 00 00 00. and KIS approach. You can see the end of search
of a 52 MB file and over 40000 hits. I've got to decode those headers and then go on from there.

After I get a little more done to the app I will push it to bitbucket.

Attached Images
Click image for larger version - Name: Screenshot_from_2014-04-18_06:21:39.png, Views: 37, Size: 454.80 KB 

__________________
andyinyakima
-----------------------------------
Recs: TBS6983, TBS5925 __ Ant: 90cm on SG6100__OSes: Linux with v4l-updatelee drivers __ RPi2 with v4l-updatelee drivers

Open Book, Open Source, You have to Open it to Know what's in it!

andyinyakima

Avatar / Picture

Registered:
Posts: 1,520
Reply with quote  #20 
Has anyone found a GS with SIS. I haven't located one. If you find one I hope it is on Ku.

All I have found is MIS streams.


Linux users.
Is anyone familiar with dvb_demux enough to adjust the code so that the Sync byte doesn't show up when
a GS stream is running. It needs to be there for TS stream.

I don't think the Sync byte (0x47) is helping with decoding a GS.

Thanks

__________________
andyinyakima
-----------------------------------
Recs: TBS6983, TBS5925 __ Ant: 90cm on SG6100__OSes: Linux with v4l-updatelee drivers __ RPi2 with v4l-updatelee drivers

Open Book, Open Source, You have to Open it to Know what's in it!
midwestmac

Registered:
Posts: 2,275
Reply with quote  #21 
Wow Andy looks like you've been hard at it.
You can try this one I think this is the kind of signal you're looking
G25 @ 93W
11749 H 8265 S2 (switching from Qpsk to 8psk) CCM Continuous, Single Input stream
This is the signal I found FX news on a while back but they change something and I don't get that now.
You might try Satmex6@113W or Satmex8@116w? I think? if I find one I'll post

__________________
Azbox Ultra, Pansat 2500, Prof7301,Tbs 6925,5980, Genpix 8psk card, Dektec 2137c, Hauppauge 950q
andyinyakima

Avatar / Picture

Registered:
Posts: 1,520
Reply with quote  #22 
Quote:
Originally Posted by midwestmac
Wow Andy looks like you've been hard at it.
You can try this one I think this is the kind of signal you're looking
G25 @ 93W
11749 H 8265 S2 (switching from Qpsk to 8psk) CCM Continuous, Single Input stream
This is the signal I found FX news on a while back but they change something and I don't get that now.
You might try Satmex6@113W or Satmex8@116w? I think? if I find one I'll post


mac,

I'm looking for an ACM with a SIS but I don't know as I will find one.

I just loaded an app on bitbucket that hopefully will help with sorting through the GS binary files:

git clone https://bitbucket.org/andyinyakima/gse_sort.git

It's in preliminary stages but I'm finding interesting "yeahs and "unexpected" finds already.

Need Qt to compile it. It works in Linux but I think it will compile in Windows also.

Tried to make it intuitive, but hey,..... my intuitive lol.

I'll go look at that signal midwestmac,
Thanks

__________________
andyinyakima
-----------------------------------
Recs: TBS6983, TBS5925 __ Ant: 90cm on SG6100__OSes: Linux with v4l-updatelee drivers __ RPi2 with v4l-updatelee drivers

Open Book, Open Source, You have to Open it to Know what's in it!
andyinyakima

Avatar / Picture

Registered:
Posts: 1,520
Reply with quote  #23 
I have attached the smallest GS I have found thus far.
The EPU is 11 bytes and 6 bytes are suppose to be a label.

Could it be an ACM command??

The first 42 is the entry BB header the end 42 is start of another BB header.

Any guesses?

 
Attached Files
txt smallGS.txt (234 Bytes, 19 views)


__________________
andyinyakima
-----------------------------------
Recs: TBS6983, TBS5925 __ Ant: 90cm on SG6100__OSes: Linux with v4l-updatelee drivers __ RPi2 with v4l-updatelee drivers

Open Book, Open Source, You have to Open it to Know what's in it!

midwestmac

Registered:
Posts: 2,275
Reply with quote  #24 
Thanks for sharing Andy I tried to compile in windows no luck I'll try again tomorrow in linux.

Looking at your txt file above thats a wierd one.  

GSE   C0 0B 00 02 00 80 3B F6 00 00 00 37 5F F5 88 88 
So the 3rd byte Frag ID is 00 whats that I wonder? With your app I was wondering if the GSE could be grouped by Frag ID?
What a mess though, there's a ton of other things like the CRC8, you mentioned then protocol type WOW! 

ADDED: Its easier said then done though nice job!
Thanks Andy

__________________
Azbox Ultra, Pansat 2500, Prof7301,Tbs 6925,5980, Genpix 8psk card, Dektec 2137c, Hauppauge 950q
andyinyakima

Avatar / Picture

Registered:
Posts: 1,520
Reply with quote  #25 
I don't know if that is a Frag ID or do they use 00 to indicate there is no Frag ID??
More observation needed I guess and more reading.
I take it mips doesn't have a satellite or least one with capability to down load GSE binaries.

I attached a BB header and GSE header search.

Hope you can get it to compile in Linux. Maybe it's a binary file issue with the Windows.
One of the reasons I like Qt is that I can cross compile Linux and Windows to a point.

 
Attached Files
txt BBHeaders_and_GSE_Headers.txt (258.66 KB, 20 views)


__________________
andyinyakima
-----------------------------------
Recs: TBS6983, TBS5925 __ Ant: 90cm on SG6100__OSes: Linux with v4l-updatelee drivers __ RPi2 with v4l-updatelee drivers

Open Book, Open Source, You have to Open it to Know what's in it!

mips

Registered:
Posts: 529
Reply with quote  #26 
Quote:
Originally Posted by andyinyakima

I take it mips doesn't have a satellite or least one with capability to down load GSE binaries.

I don't have a dish setup. I can sometimes have access to one, but it's at a remote location, so not very practical. That setup has a Prof card (no TBS card). Can I record GSE streams with a Prof card?

On a separate note, I've also been working on my own dump tool to analyze BB headers (http://www.etymonix.com/upload/BBAnalyzer-v0.01.zip). To use, I type the following at a Windows command prompt:
BBAnalyzer recording.ts > log.txt

Sample output from log.txt:
BBF @ 1319        MATYPE=$4200 [GS-C  MIS ACM ISSYI=0 GLT=0 RO=2 ISI=$00] UPL=$0000 DFL=$2C60 SYNC=$8E SYNCD=$0000 CRC1=$43

The BB header fields follow the file offset. The '$' prefix indicates hex. CRC1 indicates a valid CRC, whereas CRC0 indicates an invalid one.


@ midwestmac:

When I run the tool on your samples, I get some strange results. All the samples initially start as GS-C (GS continuous) MIS ACM, but some headers show strange results. TBSrecG1812122pt4.ts (from Nov 2013) show some invalid headers (CRC0) once in a while. Interestingly, the CRC in newer samples streamg18friday818pm.ts and streamg183-15-14911pm.ts (which I think you recorded using a different app/procedure) is always flagged as valid (there are no CRC0 entries), but what I find troubling is that some headers clearly don't make sense and are likely invalid even though they're not flagged as such.

Here's an example from the streamg18friday818pm.ts log file:
BBF @ 51617527    MATYPE=$4200 [GS-C  MIS ACM ISSYI=0 GLT=0 RO=2 ISI=$00] UPL=$0000 DFL=$2938 SYNC=$B1 SYNCD=$0000 CRC1=$82
BBF @ 51618856    MATYPE=$4200 [GS-C  MIS ACM ISSYI=0 GLT=0 RO=2 ISI=$00] UPL=$0000 DFL=$2938 SYNC=$B2 SYNCD=$0000 CRC1=$D2
BBF @ 51620185    MATYPE=$9744 [GS-HE MIS CCM ISSYI=0 GLT=1 RO=3 ISI=$44] UPL=$3850 DFL=$4B34 SYNC=$D9 SYNCD=$CC0E CRC1=$1F
BBF @ 51622601    MATYPE=$CE01 [TS    MIS ACM ISSYI=1 NPD=1 RO=2 ISI=$01] UPL=$0D51 DFL=$E4F0 SYNC=$85 SYNCD=$40F8 CRC1=$64
BBF @ 51629937    MATYPE=$4200 [GS-C  MIS ACM ISSYI=0 GLT=0 RO=2 ISI=$00] UPL=$0000 DFL=$2938 SYNC=$20 SYNCD=$0000 CRC1=$D0
BBF @ 51631266    MATYPE=$4200 [GS-C  MIS ACM ISSYI=0 GLT=0 RO=2 ISI=$00] UPL=$0000 DFL=$2938 SYNC=$21 SYNCD=$0000 CRC1=$53

All CRCs appear valid, but the 3rd and 4th headers are suspicious for multiple reasons:
- They appear to be using GS-HEM and TS formats instead of GS-C
- RO=3, an unlikely value which was reserved until the recent DVB-S2X spec
- DFL is not a multiple of 8 ($4B34)
- UPL is non-zero or not a multiple of 8 ($0D51)
- The SYNC values show a discontinuity with previous/following values

These reasons lead me to believe these headers are actually invalid. In fact, by looking at the hex bytes in that range, you can spot a few other headers that are better candidates because of the continuous SYNC value (from $18 to 1F) preceding $20, bu they didn't get reported because the DFL value of the 2nd header above was too large, causing the 3rd header to be read from the wrong place in the stream (luckily, it recovers by the 5th header).

My conclusion is that these recordings are corrupted. There are sometimes large discontinuities in the SYNC values, suggesting that some stream data is missing. That could be due to a lot of things (e.g. weak reception), but I'm puzzled by the fact that all fake BB frames report a valid CRC, almost as if the recording app (or driver?) were recalculating/overwriting the CRC blindly. It seems unlikely that the GSE streams would be transmitted with these inconsistencies.

I'm hoping you can use this tool to quickly analyze recordings on your end and perhaps find some that are error-free. Short error-free recordings are possible, but longer ones seem to always contain inconsistencies (at least all the ones I have). By comparing the output of different apps or recording modes, you might find a setup that allow you to record long GSE streams error-free.

BTW, if you come across GSE streams other than GS-C MIS ACM, I would appreciate it if you could also record those formats and upload them.

Thanks

midwestmac

Registered:
Posts: 2,275
Reply with quote  #27 
Thankyou Mips! Amazing what you find.
  I never was happy with the recording "TBSrecG1812122pt4.ts (from Nov 2013)". I used the TBSrecorder and it would have a lock on signal but as soon as I hit "record" it seemed like the signal lock would come and go.
So since then I have been using crazycats streamreader demo which seems to work a lot better. 



Quote:
Originally Posted by mips
Interestingly, the CRC in newer samples streamg18friday818pm.ts and streamg183-15-14911pm.ts (which I think you recorded using a different app/procedure) is always flagged as valid (there are no CRC0 entries), but what I find troubling is that some headers clearly don't make sense and are likely invalid even though they're not flagged as such.



These two recordings I did do different.. They were both done with crazycats program.
The  " streamg18friday818pm.ts" I believe I had the "MISfilter" set at "off" (see pic of streamreaderdemo)
Then the "streamg183-15-14911pm.ts" I believe I had it set as "0" 

I really didn't know at the time which way there to record but I believe now thats the key. So I wonder if I should leave it at "off". If you look at my crazyscan pic below it has Input stream "0,16" MIS and I don't think that "16" was there before.

As far as other recordings I'll try and get a recording of that one on g25KU it had fox news at one time and there were some fox news streams on cband too maybe one of the guys who found those can get a recording.

Thanks again for looking Mips and doing all that that had to take to take a lot of time. I've been looking at hex for a month now [smile]I'll try your app out for sure and study some moreG18GSEstreamreader.JPG G18GSE.JPG


__________________
Azbox Ultra, Pansat 2500, Prof7301,Tbs 6925,5980, Genpix 8psk card, Dektec 2137c, Hauppauge 950q
andyinyakima

Avatar / Picture

Registered:
Posts: 1,520
Reply with quote  #28 
mips, mac

What mips says I agree with, there are many areas where we don't seem to catch the flow. That is why I opted to keep it simple. If we find a set of codes that seem to jive, than build a decoder around that.
If we can get that decoder down pat we should be able to move on to other scenarios.

Does crazycat's code remove the normal TS sync byte (0x47) off the binary stream. If you view a stream with wireshark and you see the leading 0x47 on every packet than those binaries I believe are being treated as TS streams.  I believe they will cause some errors in GSE decoding.

mips you should be able to download a binary with a Prof card, but I don't have one so I'm not 100% sure.

__________________
andyinyakima
-----------------------------------
Recs: TBS6983, TBS5925 __ Ant: 90cm on SG6100__OSes: Linux with v4l-updatelee drivers __ RPi2 with v4l-updatelee drivers

Open Book, Open Source, You have to Open it to Know what's in it!
midwestmac

Registered:
Posts: 2,275
Reply with quote  #29 
@Andy- I don't think that crazycats program is removing the x47. I think it might be a matter of making a recording when a TS is flowing. Maybe he could comment. I tried to do the recordings I sent Mips by checking the signal with Transedit. Made sure there was some video flowing. But I don't know if I caught all that TS. I recorded just 2 gig's but that took an hour to record that much.
Then I go back and look at it with a hex editor and what little I know I see some internet traffic (which is easy to see in the text) and other stuff that doesnt.
So how long does a TS flow? I wonder? I have seen some on AMC15 for example that go for a while during college football with the Transedit.
Same with G18 the TS can flow for a while then stops in transedit but I'm making my recordings with crazycats program once I confirm there's some video there with Transedit. Not recording with Transedit. Its a mystery but maybe we'll get lucky some where. 


@Mips- I forgot to ask if you where able to strip away with the hex editor and piece anything together. Or was it that bad? or not worth it?

The tbsrecording I did, had a good lock on that but it took forever to hold it there and then it would lock my TBS 6925 card up to were I had to reboot. Thats why I started using crazycats program.

Thanks again


__________________
Azbox Ultra, Pansat 2500, Prof7301,Tbs 6925,5980, Genpix 8psk card, Dektec 2137c, Hauppauge 950q
mips

Registered:
Posts: 529
Reply with quote  #30 
Quote:
Originally Posted by midwestmac

  I never was happy with the recording "TBSrecG1812122pt4.ts (from Nov 2013)". I used the TBSrecorder and it would have a lock on signal but as soon as I hit "record" it seemed like the signal lock would come and go.
So since then I have been using crazycats streamreader demo which seems to work a lot better. 

The streamreader demo might work better, but the fact that all "fake" BB headers have a valid CRC is very misleading. At least with TBSrecorder, the fake BB headers had an invalid CRC, which makes sense. You'd expect fake BB headers to come up with a valid CRC once in a while (since the CRC only has 8 bits), but not always. I wonder if the streamreader app could be causing this?

Quote:
Originally Posted by midwestmac

These two recordings I did do different.. They were both done with crazycats program.
The  " streamg18friday818pm.ts" I believe I had the "MISfilter" set at "off" (see pic of streamreaderdemo)
Then the "streamg183-15-14911pm.ts" I believe I had it set as "0" 

I really didn't know at the time which way there to record but I believe now thats the key. So I wonder if I should leave it at "off".

Setting the MISfilter to off would suggest that no filtering is being done, so that's probably what I would choose. The least amount of tampering with the stream, the better.

Quote:
Originally Posted by midwestmac

As far as other recordings I'll try and get a recording of that one on g25KU it had fox news at one time and there were some fox news streams on cband too maybe one of the guys who found those can get a recording.

Thanks.

Quote:
Originally Posted by andyinyakima

Does crazycat's code remove the normal TS sync byte (0x47) off the binary stream. If you view a stream with wireshark and you see the leading 0x47 on every packet than those binaries I believe are being treated as TS streams.  I believe they will cause some errors in GSE decoding.

I don't think midwestmac has a problem with TS sync bytes (at least not in the GSE recordings I have) because I can parse large chunks of BB headers without errors. If TS sync bytes had been left behind, the DFL sizes in the BB headers would be slightly off, causing fake BB headers all over the place.

Quote:
Originally Posted by midwestmac

@Mips- I forgot to ask if you where able to strip away with the hex editor and piece anything together. Or was it that bad? or not worth it?

I didn't try it. I wanted to make sure the whole recording was clean before going any further. The valid vs invalid BB headers seem to be somewhat clustered within the file, so I could probably salvage some chunks of valid BB frames between groups of fake ones.
Previous Topic | Next Topic
Print
Reply

Quick Navigation:

Easily create a Forum Website with Website Toolbox.

Fellow Members, your posts are welcome here! Do not worry about posting everything perfect. Different receivers and LNB's will give you different Frequencies and Symbol Rates. Some set top boxes, PCI cards and USB receivers, Do Not Require all of the same information that others may need. It is not Required to post everything that others may need to tune in a feed. It is just most important to share the find. We can always adjust the Frequency and Symbol Rates and try the various Modulations and FEC's on our own receivers until we get a lock and then give a polite reply with what works for your receiver, as that information might help others as well. We all appreciate the efforts and energy of the Posters!

Thank You for Visiting GOFASTMOTORSPORTS.com - Keep Your Eyes on the Sky and the Track!